2021-06-28T22:47:46.052 INFO:root:teuthology version: 1.1.0-26631ede 2021-06-28T22:47:46.208 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi046.front.sepia.ceph.com' reimaging 2021-06-28T22:47:46.209 INFO:teuthology.lock.ops:Updating [ubuntu@smithi046.front.sepia.ceph.com]: reset os type and version on server 2021-06-28T22:47:46.210 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2021-06-28T22:47:46.228 INFO:teuthology.lock.ops:Node 'ubuntu@smithi046.front.sepia.ceph.com' reimaging is complete 2021-06-28T22:47:46.228 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging 2021-06-28T22:47:46.229 INFO:teuthology.lock.ops:Updating [ubuntu@smithi154.front.sepia.ceph.com]: reset os type and version on server 2021-06-28T22:47:46.229 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2021-06-28T22:47:46.250 INFO:teuthology.lock.ops:Node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging is complete 2021-06-28T22:47:46.250 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi184.front.sepia.ceph.com' reimaging 2021-06-28T22:47:46.251 INFO:teuthology.lock.ops:Updating [ubuntu@smithi184.front.sepia.ceph.com]: reset os type and version on server 2021-06-28T22:47:46.251 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2021-06-28T22:47:46.277 INFO:teuthology.lock.ops:Node 'ubuntu@smithi184.front.sepia.ceph.com' reimaging is complete 2021-06-28T22:47:46.465 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of rhel 8.4 2021-06-28T22:47:46.480 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of rhel 8.4 2021-06-28T22:47:46.497 INFO:teuthology.provision.fog.smithi184:Scheduling deploy of rhel 8.4 2021-06-28T22:47:46.867 INFO:teuthology.orchestra.console:Power off smithi154 2021-06-28T22:47:46.885 INFO:teuthology.orchestra.console:Power off smithi184 2021-06-28T22:47:46.901 INFO:teuthology.orchestra.console:Power off smithi046 2021-06-28T22:47:55.298 INFO:teuthology.orchestra.console:Power off for smithi154 completed 2021-06-28T22:47:55.347 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2021-06-28T22:47:55.400 INFO:teuthology.orchestra.console:Power on smithi154 2021-06-28T22:47:55.448 INFO:teuthology.orchestra.console:Power on smithi046 2021-06-28T22:47:59.448 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2021-06-28T22:47:59.550 INFO:teuthology.orchestra.console:Power on smithi184 2021-06-28T22:48:03.805 INFO:teuthology.orchestra.console:Power on for smithi154 completed 2021-06-28T22:48:03.832 INFO:teuthology.orchestra.console:Power on for smithi184 completed 2021-06-28T22:48:03.855 INFO:teuthology.orchestra.console:Power on for smithi046 completed 2021-06-28T22:48:03.906 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish 2021-06-28T22:48:03.941 INFO:teuthology.provision.fog.smithi184:Waiting for deploy to finish 2021-06-28T22:48:03.957 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish 2021-06-28T22:52:25.423 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-28T22:52:35.701 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-28T22:52:35.834 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-28T22:52:35.916 INFO:teuthology.provision.fog.smithi046:Node is ready 2021-06-28T22:52:35.975 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com 2021-06-28T22:52:36.031 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2021-06-28T22:52:36.048 INFO:teuthology.provision.fog.smithi184:Node is ready 2021-06-28T22:52:36.066 INFO:teuthology.orchestra.run.smithi184.stdout:smithi184.front.sepia.ceph.com 2021-06-28T22:52:36.123 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.15.184 smithi184.front.sepia.ceph.com smithi184 2021-06-28T22:52:36.370 INFO:teuthology.provision.fog.smithi046:Deploy complete! 2021-06-28T22:52:36.482 INFO:teuthology.provision.fog.smithi184:Deploy complete! 2021-06-28T22:52:46.084 INFO:teuthology.provision.fog.smithi154:Node is ready 2021-06-28T22:52:46.104 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com 2021-06-28T22:52:46.162 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154 2021-06-28T22:52:46.506 INFO:teuthology.provision.fog.smithi154:Deploy complete! 2021-06-28T22:52:46.682 INFO:teuthology.lock.ops:Checking smithi046.front.sepia.ceph.com 2021-06-28T22:52:46.683 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-28T22:52:46.700 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com 2021-06-28T22:52:46.701 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-28T22:52:46.716 INFO:teuthology.lock.ops:Checking smithi184.front.sepia.ceph.com 2021-06-28T22:52:46.717 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-28T22:52:46.733 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: set os type and version on server 2021-06-28T22:52:46.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-28T22:52:46.974 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2021-06-28T22:52:47.030 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Red Hat Enterprise Linux" 2021-06-28T22:52:47.031 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="8.4 (Ootpa)" 2021-06-28T22:52:47.031 INFO:teuthology.orchestra.run.smithi046.stdout:ID="rhel" 2021-06-28T22:52:47.031 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="fedora" 2021-06-28T22:52:47.031 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="8.4" 2021-06-28T22:52:47.032 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el8" 2021-06-28T22:52:47.032 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-06-28T22:52:47.032 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2021-06-28T22:52:47.033 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-06-28T22:52:47.033 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.redhat.com/" 2021-06-28T22:52:47.033 INFO:teuthology.orchestra.run.smithi046.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-06-28T22:52:47.033 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-06-28T22:52:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: 2021-06-28T22:52:47.034 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-06-28T22:52:47.034 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-06-28T22:52:47.034 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-06-28T22:52:47.035 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-06-28T22:52:47.037 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2021-06-28T22:52:47.055 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server 2021-06-28T22:52:47.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-28T22:52:47.249 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2021-06-28T22:52:47.306 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Red Hat Enterprise Linux" 2021-06-28T22:52:47.306 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8.4 (Ootpa)" 2021-06-28T22:52:47.306 INFO:teuthology.orchestra.run.smithi154.stdout:ID="rhel" 2021-06-28T22:52:47.307 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="fedora" 2021-06-28T22:52:47.307 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8.4" 2021-06-28T22:52:47.307 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8" 2021-06-28T22:52:47.307 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-06-28T22:52:47.308 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2021-06-28T22:52:47.308 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-06-28T22:52:47.308 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.redhat.com/" 2021-06-28T22:52:47.308 INFO:teuthology.orchestra.run.smithi154.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-06-28T22:52:47.309 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-06-28T22:52:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: 2021-06-28T22:52:47.309 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-06-28T22:52:47.309 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-06-28T22:52:47.309 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-06-28T22:52:47.310 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-06-28T22:52:47.310 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2021-06-28T22:52:47.325 INFO:teuthology.lock.ops:Updating [smithi184.front.sepia.ceph.com]: set os type and version on server 2021-06-28T22:52:47.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-28T22:52:47.561 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2021-06-28T22:52:47.615 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="Red Hat Enterprise Linux" 2021-06-28T22:52:47.616 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="8.4 (Ootpa)" 2021-06-28T22:52:47.616 INFO:teuthology.orchestra.run.smithi184.stdout:ID="rhel" 2021-06-28T22:52:47.616 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="fedora" 2021-06-28T22:52:47.616 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="8.4" 2021-06-28T22:52:47.616 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el8" 2021-06-28T22:52:47.617 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-06-28T22:52:47.617 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2021-06-28T22:52:47.617 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-06-28T22:52:47.617 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://www.redhat.com/" 2021-06-28T22:52:47.618 INFO:teuthology.orchestra.run.smithi184.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-06-28T22:52:47.618 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-06-28T22:52:47.618 INFO:teuthology.orchestra.run.smithi184.stdout: 2021-06-28T22:52:47.618 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-06-28T22:52:47.619 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-06-28T22:52:47.619 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-06-28T22:52:47.619 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-06-28T22:52:47.619 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2021-06-28T22:52:48.793 INFO:teuthology.dispatcher.supervisor:Running job 6240006 2021-06-28T22:52:48.794 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-06-28_22:30:48-fs-master-distro-basic-smithi/6240006 --name pdonnell-2021-06-28_22:30:48-fs-master-distro-basic-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} -- /home/teuthworker/archive/pdonnell-2021-06-28_22:30:48-fs-master-distro-basic-smithi/6240006/orig.config.yaml 2021-06-28T22:52:48.799 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-06-28_22:30:48-fs-master-distro-basic-smithi/6240006 2021-06-28T22:52:48.801 INFO:teuthology.dispatcher.supervisor:Job PID: 12676 2021-06-28T22:52:48.801 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-28T23:02:49.022 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-28T23:02:49.193 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-28T23:02:49.194 INFO:teuthology.nuke:Checking targets against current locks 2021-06-28T23:02:49.290 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-28T23:02:49.294 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-28T23:02:49.296 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-28T23:02:49.357 INFO:teuthology.orchestra.console:Power off smithi046 2021-06-28T23:02:49.372 INFO:teuthology.orchestra.console:Power off smithi184 2021-06-28T23:02:49.383 INFO:teuthology.orchestra.console:Power off smithi154 2021-06-28T23:02:53.628 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2021-06-28T23:02:53.633 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2021-06-28T23:02:53.639 INFO:teuthology.orchestra.console:Power off for smithi154 completed 2021-06-28T23:02:53.764 ERROR:teuthology.lock.ops:failed to unlock smithi154.front.sepia.ceph.com. reason: Cannot unlock an already-unlocked node 2021-06-28T23:02:53.767 INFO:teuthology.lock.ops:unlocked smithi046.front.sepia.ceph.com 2021-06-28T23:02:53.771 INFO:teuthology.lock.ops:unlocked smithi184.front.sepia.ceph.com