2021-07-23T16:25:02.064 INFO:root:teuthology version: 1.1.0-b6f3efab 2021-07-23T16:25:02.209 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi045.front.sepia.ceph.com' reimaging 2021-07-23T16:25:02.210 INFO:teuthology.lock.ops:Updating [ubuntu@smithi045.front.sepia.ceph.com]: reset os type and version on server 2021-07-23T16:25:02.210 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2021-07-23T16:25:02.232 INFO:teuthology.lock.ops:Node 'ubuntu@smithi045.front.sepia.ceph.com' reimaging is complete 2021-07-23T16:25:02.232 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi075.front.sepia.ceph.com' reimaging 2021-07-23T16:25:02.232 INFO:teuthology.lock.ops:Updating [ubuntu@smithi075.front.sepia.ceph.com]: reset os type and version on server 2021-07-23T16:25:02.233 INFO:teuthology.lock.ops:Updating smithi075.front.sepia.ceph.com on lock server 2021-07-23T16:25:02.248 INFO:teuthology.lock.ops:Node 'ubuntu@smithi075.front.sepia.ceph.com' reimaging is complete 2021-07-23T16:25:02.248 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi129.front.sepia.ceph.com' reimaging 2021-07-23T16:25:02.248 INFO:teuthology.lock.ops:Updating [ubuntu@smithi129.front.sepia.ceph.com]: reset os type and version on server 2021-07-23T16:25:02.249 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2021-07-23T16:25:02.264 INFO:teuthology.lock.ops:Node 'ubuntu@smithi129.front.sepia.ceph.com' reimaging is complete 2021-07-23T16:25:02.470 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of rhel 8.4 2021-07-23T16:25:02.507 INFO:teuthology.provision.fog.smithi075:Scheduling deploy of rhel 8.4 2021-07-23T16:25:02.510 INFO:teuthology.provision.fog.smithi129:Scheduling deploy of rhel 8.4 2021-07-23T16:25:03.050 INFO:teuthology.orchestra.console:Power off smithi045 2021-07-23T16:25:03.101 INFO:teuthology.orchestra.console:Power off smithi129 2021-07-23T16:25:03.142 INFO:teuthology.orchestra.console:Power off smithi075 2021-07-23T16:25:11.445 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2021-07-23T16:25:11.485 INFO:teuthology.orchestra.console:Power off for smithi129 completed 2021-07-23T16:25:11.525 INFO:teuthology.orchestra.console:Power off for smithi075 completed 2021-07-23T16:25:11.863 INFO:teuthology.orchestra.console:Power on smithi045 2021-07-23T16:25:11.877 INFO:teuthology.orchestra.console:Power on smithi129 2021-07-23T16:25:11.964 INFO:teuthology.orchestra.console:Power on smithi075 2021-07-23T16:25:16.229 INFO:teuthology.orchestra.console:Power on for smithi045 completed 2021-07-23T16:25:16.275 INFO:teuthology.orchestra.console:Power on for smithi075 completed 2021-07-23T16:25:16.331 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish 2021-07-23T16:25:16.376 INFO:teuthology.provision.fog.smithi075:Waiting for deploy to finish 2021-07-23T16:25:20.368 INFO:teuthology.orchestra.console:Power on for smithi129 completed 2021-07-23T16:25:20.469 INFO:teuthology.provision.fog.smithi129:Waiting for deploy to finish 2021-07-23T16:29:25.103 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-23T16:29:25.323 INFO:teuthology.provision.fog.smithi045:Node is ready 2021-07-23T16:29:25.339 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com 2021-07-23T16:29:25.397 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045 2021-07-23T16:29:25.735 INFO:teuthology.provision.fog.smithi045:Deploy complete! 2021-07-23T16:29:29.378 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-23T16:29:29.806 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-23T16:29:49.751 INFO:teuthology.provision.fog.smithi129:Node is ready 2021-07-23T16:29:49.770 INFO:teuthology.orchestra.run.smithi129.stdout:smithi129.front.sepia.ceph.com 2021-07-23T16:29:49.827 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129 smithi129.front.sepia.ceph.com smithi129 2021-07-23T16:29:49.974 INFO:teuthology.provision.fog.smithi075:Node is ready 2021-07-23T16:29:49.993 INFO:teuthology.orchestra.run.smithi075.stdout:smithi075.front.sepia.ceph.com 2021-07-23T16:29:50.051 INFO:teuthology.orchestra.run.smithi075.stdout:172.21.15.75 smithi075.front.sepia.ceph.com smithi075 2021-07-23T16:29:50.167 INFO:teuthology.provision.fog.smithi129:Deploy complete! 2021-07-23T16:29:50.393 INFO:teuthology.provision.fog.smithi075:Deploy complete! 2021-07-23T16:29:50.539 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com 2021-07-23T16:29:50.541 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-23T16:29:50.558 INFO:teuthology.lock.ops:Checking smithi075.front.sepia.ceph.com 2021-07-23T16:29:50.558 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-23T16:29:50.572 INFO:teuthology.lock.ops:Checking smithi129.front.sepia.ceph.com 2021-07-23T16:29:50.573 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-23T16:29:50.588 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server 2021-07-23T16:29:50.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-23T16:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2021-07-23T16:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Red Hat Enterprise Linux" 2021-07-23T16:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8.4 (Ootpa)" 2021-07-23T16:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:ID="rhel" 2021-07-23T16:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="fedora" 2021-07-23T16:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8.4" 2021-07-23T16:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2021-07-23T16:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-07-23T16:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2021-07-23T16:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-07-23T16:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.redhat.com/" 2021-07-23T16:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-07-23T16:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-23T16:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: 2021-07-23T16:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-07-23T16:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-07-23T16:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-07-23T16:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-07-23T16:29:50.897 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2021-07-23T16:29:50.913 INFO:teuthology.lock.ops:Updating [smithi075.front.sepia.ceph.com]: set os type and version on server 2021-07-23T16:29:50.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-23T16:29:51.101 INFO:teuthology.orchestra.run.smithi075.stdout:x86_64 2021-07-23T16:29:51.160 INFO:teuthology.orchestra.run.smithi075.stdout:NAME="Red Hat Enterprise Linux" 2021-07-23T16:29:51.160 INFO:teuthology.orchestra.run.smithi075.stdout:VERSION="8.4 (Ootpa)" 2021-07-23T16:29:51.160 INFO:teuthology.orchestra.run.smithi075.stdout:ID="rhel" 2021-07-23T16:29:51.161 INFO:teuthology.orchestra.run.smithi075.stdout:ID_LIKE="fedora" 2021-07-23T16:29:51.161 INFO:teuthology.orchestra.run.smithi075.stdout:VERSION_ID="8.4" 2021-07-23T16:29:51.162 INFO:teuthology.orchestra.run.smithi075.stdout:PLATFORM_ID="platform:el8" 2021-07-23T16:29:51.162 INFO:teuthology.orchestra.run.smithi075.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-07-23T16:29:51.163 INFO:teuthology.orchestra.run.smithi075.stdout:ANSI_COLOR="0;31" 2021-07-23T16:29:51.163 INFO:teuthology.orchestra.run.smithi075.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-07-23T16:29:51.163 INFO:teuthology.orchestra.run.smithi075.stdout:HOME_URL="https://www.redhat.com/" 2021-07-23T16:29:51.163 INFO:teuthology.orchestra.run.smithi075.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-07-23T16:29:51.164 INFO:teuthology.orchestra.run.smithi075.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-23T16:29:51.164 INFO:teuthology.orchestra.run.smithi075.stdout: 2021-07-23T16:29:51.164 INFO:teuthology.orchestra.run.smithi075.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-07-23T16:29:51.164 INFO:teuthology.orchestra.run.smithi075.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-07-23T16:29:51.165 INFO:teuthology.orchestra.run.smithi075.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-07-23T16:29:51.165 INFO:teuthology.orchestra.run.smithi075.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-07-23T16:29:51.165 INFO:teuthology.lock.ops:Updating smithi075.front.sepia.ceph.com on lock server 2021-07-23T16:29:51.181 INFO:teuthology.lock.ops:Updating [smithi129.front.sepia.ceph.com]: set os type and version on server 2021-07-23T16:29:51.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-23T16:29:51.379 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2021-07-23T16:29:51.437 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="Red Hat Enterprise Linux" 2021-07-23T16:29:51.437 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="8.4 (Ootpa)" 2021-07-23T16:29:51.437 INFO:teuthology.orchestra.run.smithi129.stdout:ID="rhel" 2021-07-23T16:29:51.438 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="fedora" 2021-07-23T16:29:51.438 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="8.4" 2021-07-23T16:29:51.438 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el8" 2021-07-23T16:29:51.438 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-07-23T16:29:51.439 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2021-07-23T16:29:51.439 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-07-23T16:29:51.439 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://www.redhat.com/" 2021-07-23T16:29:51.439 INFO:teuthology.orchestra.run.smithi129.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-07-23T16:29:51.439 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-23T16:29:51.440 INFO:teuthology.orchestra.run.smithi129.stdout: 2021-07-23T16:29:51.440 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-07-23T16:29:51.440 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-07-23T16:29:51.440 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-07-23T16:29:51.441 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-07-23T16:29:51.441 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2021-07-23T16:29:51.530 INFO:teuthology.dispatcher.supervisor:Running job 6288802 2021-07-23T16:29:51.531 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-07-23_15:30:01-fs:workload-master-distro-basic-smithi/6288802 --name pdonnell-2021-07-23_15:30:01-fs:workload-master-distro-basic-smithi --description fs:workload/{begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 rhel_8 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} -- /home/teuthworker/archive/pdonnell-2021-07-23_15:30:01-fs:workload-master-distro-basic-smithi/6288802/orig.config.yaml 2021-07-23T16:29:51.535 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-07-23_15:30:01-fs:workload-master-distro-basic-smithi/6288802 2021-07-23T16:29:51.536 INFO:teuthology.dispatcher.supervisor:Job PID: 24395 2021-07-23T16:29:51.536 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-07-23T22:54:00.206 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/pdonnell-2021-07-23_15:30:01-fs:workload-master-distro-basic-smithi/jobs/ failed with status 500: Internal Server Error

Internal Server Error

2021-07-24T04:30:10.515 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-07-24T04:30:10.736 INFO:teuthology.kill:Killing Pids: {24395} 2021-07-24T04:30:11.760 INFO:teuthology.kill:Nuking machines: ['smithi045', 'smithi075', 'smithi129'] 2021-07-24T04:30:12.646 INFO:teuthology.kill:2021-07-24 04:30:12,645.645 INFO:teuthology.nuke:targets: 2021-07-24T04:30:12.647 INFO:teuthology.kill: smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkSqlu/YUJygGU9miN5Z1nAv0vENeis4P7hzkWIRaDDXA9BcmdugFL7mnsD9saq22ASo4Uw4laF9gq6644izfef5GAHozGg7Wsw5c98Y2wiW7ezq6IdyFiJW2c7azGoJ9OrnGLkIvItHPCfUctRJn+mI0PxtfW0LbvhwKLWwWd3td/d52OgtF4LEWRF6V35a7dZzqcUXwP2H5+mPrBBMBIdTj1sPkeKDN091B4VGtx6aTdLl2tdjJqCAfsRJcLsDOz4Pk1uGWCa7Tt59sY4aN2T4H5qSop6GIagxRtE9qbafQOPiJtrIOn4/KV3sAqszETDl/TMVC9O8/AkePzYuiDhU6P+Rr/dVfXf9ja72duKSMgv1MSHrexqEHIniLazYRD2hs5pa4/+LcZrM8rZZgAwb7GkbndgISjzGzP7PADRm5zpWk1dJuhkdTlTE7d4UxtvSLkQPYSBoKLks3IZKgg0zeaPqQGuIj4ReafG89X7DSoqb7M2uzX8YkzrQxNa6M= 2021-07-24T04:30:12.647 INFO:teuthology.kill: smithi075.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDihw81En6m3yThbBnUnuss+scSNd6t1D4g/6C6w5kt46j0P02X260f43Bkw2/gEm4KuSOtHXyBCWxVl61tZrQ+p53r586Kfk7f+LLOePMS/bUWg8AvpZ4ZraQhe2prsBUnmui+n16SE4WsIZn8TcFICDx0G5uFmVcLAVn9Z5mzYhb+U1eVQa5jioHCUN8vxYlzHiIXXpsJp8q6b6kKGvktKwvmhl7UXcwmx4frQVVL8Jb8LXYXhb4+52GcCkXMMCxPvlSuQAR+2XAuB9mQBPtrs5gKhWrWUxCsNN2dOfBcgVntEFxT3RAGOK/9bG8nPNpdnrSPc5YCnqTI1Zg5a0A7RU+eLb4odTt0AWu1kZl53AVMtbZEIpG6dAjbQxUodVHqYZ4ShEzq1K8aLFzanuaUvdCQY8f5uyfK5bDuUdyhoxYaWD9m+tkUj0v3cCPeuqxUzCX4m4yy9xxiGSvjb9jisxvvclCf+XPYyXCULtbYOyh5ANsnWAxRJLlBmfSYcO0= 2021-07-24T04:30:12.647 INFO:teuthology.kill: smithi129.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHiOvFERxDym455+XHuOIZ6w0wwQVBtnzH8zbQnwg+UvkBX2XRptgX69pqs8XTlhzq+5J1yiqQCemNUhsdnj1F3b8mx2F8EZeLoGhV3rt0uPmcRHogbm7dyopCJTc743g0XxK7yyA1KKSF+g75IQ500JMgtCX9QJ6Y4z1c5nDASnzVL8Dn91QIvKhQaORpmRA6zexfUBVFET3kfYw5lcu1er7hre3ajCpDp4DN8mmY5NuL/2lzF1POgaVe6KMmMJEFoqCkcfl+gP4fhxNSrIY47A8KLHoDGRmIVhctN91wUqTXXRElhGTsrNtQIIQ0IUb9gQZVDt1Nz60Eso20d3nK9YiI1H3JqAqteGCWMpyDK3TQkMDGanIQnf+E8PwnVIddZw4Dxw6dUBGO6TXGcXIAqWxS+wkDMMq3DSIVcN89GZqMAf0pKmPTl7HxO7z23zw+7s+aH85V0stHVIqR4Yden8Skilz/Y/WVbG4qJEqAkT9aWoAY+qo9/tym2p389M= 2021-07-24T04:30:12.647 INFO:teuthology.kill:2021-07-24 04:30:12,646.646 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-24T04:30:12.648 INFO:teuthology.kill:2021-07-24 04:30:12,648.648 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-24T04:30:12.649 INFO:teuthology.kill:2021-07-24 04:30:12,649.649 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-24T04:30:12.710 INFO:teuthology.kill:2021-07-24 04:30:12,709.709 INFO:teuthology.orchestra.console:Power off smithi075 2021-07-24T04:30:12.722 INFO:teuthology.kill:2021-07-24 04:30:12,721.721 INFO:teuthology.orchestra.console:Power off smithi045 2021-07-24T04:30:12.732 INFO:teuthology.kill:2021-07-24 04:30:12,732.732 INFO:teuthology.orchestra.console:Power off smithi129 2021-07-24T04:30:21.097 INFO:teuthology.kill:2021-07-24 04:30:21,096.096 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2021-07-24T04:30:25.205 INFO:teuthology.kill:2021-07-24 04:30:25,204.204 INFO:teuthology.orchestra.console:Power off for smithi075 completed 2021-07-24T04:30:25.211 INFO:teuthology.kill:2021-07-24 04:30:25,211.211 INFO:teuthology.orchestra.console:Power off for smithi129 completed 2021-07-24T04:30:25.429 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2021-07-24T04:30:25.429 INFO:teuthology.task.internal:roles: ubuntu@smithi075.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2021-07-24T04:30:25.429 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2021-07-24T04:30:25.430 INFO:teuthology.misc:Compressing logs... 2021-07-24T04:30:25.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-24T04:31:25.470 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-07-24T04:31:25.534 INFO:teuthology.kill:No teuthology processes running 2021-07-24T04:31:26.453 INFO:teuthology.kill:Nuking machines: ['smithi045', 'smithi075', 'smithi129'] 2021-07-24T04:31:27.281 INFO:teuthology.kill:2021-07-24 04:31:27,281.281 INFO:teuthology.nuke:targets: 2021-07-24T04:31:27.282 INFO:teuthology.kill: smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkSqlu/YUJygGU9miN5Z1nAv0vENeis4P7hzkWIRaDDXA9BcmdugFL7mnsD9saq22ASo4Uw4laF9gq6644izfef5GAHozGg7Wsw5c98Y2wiW7ezq6IdyFiJW2c7azGoJ9OrnGLkIvItHPCfUctRJn+mI0PxtfW0LbvhwKLWwWd3td/d52OgtF4LEWRF6V35a7dZzqcUXwP2H5+mPrBBMBIdTj1sPkeKDN091B4VGtx6aTdLl2tdjJqCAfsRJcLsDOz4Pk1uGWCa7Tt59sY4aN2T4H5qSop6GIagxRtE9qbafQOPiJtrIOn4/KV3sAqszETDl/TMVC9O8/AkePzYuiDhU6P+Rr/dVfXf9ja72duKSMgv1MSHrexqEHIniLazYRD2hs5pa4/+LcZrM8rZZgAwb7GkbndgISjzGzP7PADRm5zpWk1dJuhkdTlTE7d4UxtvSLkQPYSBoKLks3IZKgg0zeaPqQGuIj4ReafG89X7DSoqb7M2uzX8YkzrQxNa6M= 2021-07-24T04:31:27.283 INFO:teuthology.kill: smithi075.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDihw81En6m3yThbBnUnuss+scSNd6t1D4g/6C6w5kt46j0P02X260f43Bkw2/gEm4KuSOtHXyBCWxVl61tZrQ+p53r586Kfk7f+LLOePMS/bUWg8AvpZ4ZraQhe2prsBUnmui+n16SE4WsIZn8TcFICDx0G5uFmVcLAVn9Z5mzYhb+U1eVQa5jioHCUN8vxYlzHiIXXpsJp8q6b6kKGvktKwvmhl7UXcwmx4frQVVL8Jb8LXYXhb4+52GcCkXMMCxPvlSuQAR+2XAuB9mQBPtrs5gKhWrWUxCsNN2dOfBcgVntEFxT3RAGOK/9bG8nPNpdnrSPc5YCnqTI1Zg5a0A7RU+eLb4odTt0AWu1kZl53AVMtbZEIpG6dAjbQxUodVHqYZ4ShEzq1K8aLFzanuaUvdCQY8f5uyfK5bDuUdyhoxYaWD9m+tkUj0v3cCPeuqxUzCX4m4yy9xxiGSvjb9jisxvvclCf+XPYyXCULtbYOyh5ANsnWAxRJLlBmfSYcO0= 2021-07-24T04:31:27.283 INFO:teuthology.kill: smithi129.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHiOvFERxDym455+XHuOIZ6w0wwQVBtnzH8zbQnwg+UvkBX2XRptgX69pqs8XTlhzq+5J1yiqQCemNUhsdnj1F3b8mx2F8EZeLoGhV3rt0uPmcRHogbm7dyopCJTc743g0XxK7yyA1KKSF+g75IQ500JMgtCX9QJ6Y4z1c5nDASnzVL8Dn91QIvKhQaORpmRA6zexfUBVFET3kfYw5lcu1er7hre3ajCpDp4DN8mmY5NuL/2lzF1POgaVe6KMmMJEFoqCkcfl+gP4fhxNSrIY47A8KLHoDGRmIVhctN91wUqTXXRElhGTsrNtQIIQ0IUb9gQZVDt1Nz60Eso20d3nK9YiI1H3JqAqteGCWMpyDK3TQkMDGanIQnf+E8PwnVIddZw4Dxw6dUBGO6TXGcXIAqWxS+wkDMMq3DSIVcN89GZqMAf0pKmPTl7HxO7z23zw+7s+aH85V0stHVIqR4Yden8Skilz/Y/WVbG4qJEqAkT9aWoAY+qo9/tym2p389M= 2021-07-24T04:31:27.303 INFO:teuthology.kill:2021-07-24 04:31:27,303.303 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-24T04:31:27.306 INFO:teuthology.kill:2021-07-24 04:31:27,306.306 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-24T04:31:27.307 INFO:teuthology.kill:2021-07-24 04:31:27,307.307 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-24T04:31:27.365 INFO:teuthology.kill:2021-07-24 04:31:27,365.365 INFO:teuthology.orchestra.console:Power off smithi129 2021-07-24T04:31:27.379 INFO:teuthology.kill:2021-07-24 04:31:27,378.378 INFO:teuthology.orchestra.console:Power off smithi075 2021-07-24T04:31:27.389 INFO:teuthology.kill:2021-07-24 04:31:27,389.389 INFO:teuthology.orchestra.console:Power off smithi045 2021-07-24T04:31:27.516 INFO:teuthology.kill:2021-07-24 04:31:27,516.516 INFO:teuthology.orchestra.console:Power off for smithi129 completed 2021-07-24T04:31:27.520 INFO:teuthology.kill:2021-07-24 04:31:27,520.520 INFO:teuthology.orchestra.console:Power off for smithi075 completed 2021-07-24T04:31:27.529 INFO:teuthology.kill:2021-07-24 04:31:27,528.528 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2021-07-24T04:31:27.662 INFO:teuthology.kill:2021-07-24 04:31:27,661.661 INFO:teuthology.lock.ops:unlocked smithi129.front.sepia.ceph.com 2021-07-24T04:31:27.666 INFO:teuthology.kill:2021-07-24 04:31:27,666.666 INFO:teuthology.lock.ops:unlocked smithi045.front.sepia.ceph.com 2021-07-24T04:31:27.667 INFO:teuthology.kill:2021-07-24 04:31:27,666.666 INFO:teuthology.lock.ops:unlocked smithi075.front.sepia.ceph.com 2021-07-24T04:33:27.861 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15