2021-06-10T01:51:05.301 INFO:root:teuthology version: 1.1.0-f359b10d
2021-06-10T01:51:05.435 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging
2021-06-10T01:51:05.437 INFO:teuthology.lock.ops:Updating [ubuntu@smithi092.front.sepia.ceph.com]: reset os type and version on server
2021-06-10T01:51:05.437 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server
2021-06-10T01:51:05.466 INFO:teuthology.lock.ops:Node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging is complete
2021-06-10T01:51:05.467 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging
2021-06-10T01:51:05.467 INFO:teuthology.lock.ops:Updating [ubuntu@smithi106.front.sepia.ceph.com]: reset os type and version on server
2021-06-10T01:51:05.467 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2021-06-10T01:51:05.487 INFO:teuthology.lock.ops:Node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging is complete
2021-06-10T01:51:05.487 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging
2021-06-10T01:51:05.487 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server
2021-06-10T01:51:05.488 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2021-06-10T01:51:05.501 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete
2021-06-10T01:51:05.667 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of centos 8.3
2021-06-10T01:51:05.688 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of centos 8.3
2021-06-10T01:51:05.720 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 8.3
2021-06-10T01:51:06.020 INFO:teuthology.orchestra.console:Power off smithi106
2021-06-10T01:51:06.040 INFO:teuthology.orchestra.console:Power off smithi110
2021-06-10T01:51:06.053 INFO:teuthology.orchestra.console:Power off smithi092
2021-06-10T01:51:18.524 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-06-10T01:51:18.531 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2021-06-10T01:51:18.541 INFO:teuthology.orchestra.console:Power off for smithi092 completed
2021-06-10T01:51:18.626 INFO:teuthology.orchestra.console:Power on smithi110
2021-06-10T01:51:18.637 INFO:teuthology.orchestra.console:Power on smithi106
2021-06-10T01:51:18.647 INFO:teuthology.orchestra.console:Power on smithi092
2021-06-10T01:51:22.898 INFO:teuthology.orchestra.console:Power on for smithi110 completed
2021-06-10T01:51:22.903 INFO:teuthology.orchestra.console:Power on for smithi106 completed
2021-06-10T01:51:22.909 INFO:teuthology.orchestra.console:Power on for smithi092 completed
2021-06-10T01:51:22.999 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish
2021-06-10T01:51:23.004 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish
2021-06-10T01:51:23.010 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish
2021-06-10T01:55:54.984 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-06-10T01:55:55.308 INFO:teuthology.provision.fog.smithi110:Node is ready
2021-06-10T01:55:55.323 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com
2021-06-10T01:55:55.378 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110
2021-06-10T01:55:55.690 INFO:teuthology.provision.fog.smithi110:Deploy complete!
2021-06-10T01:56:10.216 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-06-10T01:56:10.311 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-06-10T01:56:10.525 INFO:teuthology.provision.fog.smithi106:Node is ready
2021-06-10T01:56:10.540 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com
2021-06-10T01:56:10.593 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106
2021-06-10T01:56:10.643 INFO:teuthology.provision.fog.smithi092:Node is ready
2021-06-10T01:56:10.659 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com
2021-06-10T01:56:10.715 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092
2021-06-10T01:56:10.910 INFO:teuthology.provision.fog.smithi106:Deploy complete!
2021-06-10T01:56:11.050 INFO:teuthology.provision.fog.smithi092:Deploy complete!
2021-06-10T01:56:11.221 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com
2021-06-10T01:56:11.222 INFO:teuthology.lock.ops:New key found. Updating...
2021-06-10T01:56:11.240 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com
2021-06-10T01:56:11.240 INFO:teuthology.lock.ops:New key found. Updating...
2021-06-10T01:56:11.255 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com
2021-06-10T01:56:11.255 INFO:teuthology.lock.ops:New key found. Updating...
2021-06-10T01:56:11.273 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server
2021-06-10T01:56:11.274 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-10T01:56:11.551 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64
2021-06-10T01:56:11.606 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Linux"
2021-06-10T01:56:11.606 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8"
2021-06-10T01:56:11.607 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos"
2021-06-10T01:56:11.607 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora"
2021-06-10T01:56:11.607 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8"
2021-06-10T01:56:11.607 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8"
2021-06-10T01:56:11.608 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Linux 8"
2021-06-10T01:56:11.608 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31"
2021-06-10T01:56:11.608 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-06-10T01:56:11.609 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/"
2021-06-10T01:56:11.610 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-06-10T01:56:11.610 INFO:teuthology.orchestra.run.smithi092.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-06-10T01:56:11.611 INFO:teuthology.orchestra.run.smithi092.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-06-10T01:56:11.613 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server
2021-06-10T01:56:11.632 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server
2021-06-10T01:56:11.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-10T01:56:11.919 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64
2021-06-10T01:56:11.974 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Linux"
2021-06-10T01:56:11.974 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8"
2021-06-10T01:56:11.974 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos"
2021-06-10T01:56:11.975 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora"
2021-06-10T01:56:11.975 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8"
2021-06-10T01:56:11.975 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8"
2021-06-10T01:56:11.975 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Linux 8"
2021-06-10T01:56:11.976 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31"
2021-06-10T01:56:11.976 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-06-10T01:56:11.976 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/"
2021-06-10T01:56:11.976 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-06-10T01:56:11.976 INFO:teuthology.orchestra.run.smithi106.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-06-10T01:56:11.977 INFO:teuthology.orchestra.run.smithi106.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-06-10T01:56:11.977 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2021-06-10T01:56:11.992 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server
2021-06-10T01:56:11.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-10T01:56:12.279 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64
2021-06-10T01:56:12.334 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Linux"
2021-06-10T01:56:12.334 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8"
2021-06-10T01:56:12.335 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos"
2021-06-10T01:56:12.335 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora"
2021-06-10T01:56:12.335 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8"
2021-06-10T01:56:12.336 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8"
2021-06-10T01:56:12.336 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Linux 8"
2021-06-10T01:56:12.336 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2021-06-10T01:56:12.336 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-06-10T01:56:12.337 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/"
2021-06-10T01:56:12.337 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-06-10T01:56:12.337 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-06-10T01:56:12.337 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-06-10T01:56:12.338 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2021-06-10T01:56:12.423 INFO:teuthology.dispatcher.supervisor:Running job 6163578
2021-06-10T01:56:12.424 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-06-09_23:20:51-fs-wip-pdonnell-testing-20210608.022222-distro-basic-smithi/6163578 --name pdonnell-2021-06-09_23:20:51-fs-wip-pdonnell-testing-20210608.022222-distro-basic-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd}} -- /home/teuthworker/archive/pdonnell-2021-06-09_23:20:51-fs-wip-pdonnell-testing-20210608.022222-distro-basic-smithi/6163578/orig.config.yaml
2021-06-10T01:56:12.427 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-06-09_23:20:51-fs-wip-pdonnell-testing-20210608.022222-distro-basic-smithi/6163578
2021-06-10T01:56:12.428 INFO:teuthology.dispatcher.supervisor:Job PID: 18770
2021-06-10T01:56:12.429 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-06-10T02:14:13.609 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-06-10T02:14:13.772 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-06-10T02:14:13.772 INFO:teuthology.nuke:Checking targets against current locks
2021-06-10T02:14:13.868 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-10T02:14:13.871 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-10T02:14:13.873 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-10T02:14:13.932 INFO:teuthology.orchestra.console:Power off smithi106
2021-06-10T02:14:13.946 INFO:teuthology.orchestra.console:Power off smithi110
2021-06-10T02:14:13.957 INFO:teuthology.orchestra.console:Power off smithi092
2021-06-10T02:14:22.309 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2021-06-10T02:14:22.315 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-06-10T02:14:22.321 INFO:teuthology.orchestra.console:Power off for smithi092 completed
2021-06-10T02:14:22.445 ERROR:teuthology.lock.ops:failed to unlock smithi092.front.sepia.ceph.com. reason: Cannot unlock an already-unlocked node
2021-06-10T02:14:22.448 INFO:teuthology.lock.ops:unlocked smithi106.front.sepia.ceph.com
2021-06-10T02:14:22.449 ERROR:teuthology.lock.ops:failed to unlock smithi110.front.sepia.ceph.com. reason: 500