2021-08-03T19:45:17.064 INFO:root:teuthology version: 1.1.0-73aa7e3b 2021-08-03T19:45:17.154 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging 2021-08-03T19:45:17.156 INFO:teuthology.lock.ops:Updating [ubuntu@smithi149.front.sepia.ceph.com]: reset os type and version on server 2021-08-03T19:45:17.156 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2021-08-03T19:45:17.172 INFO:teuthology.lock.ops:Node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging is complete 2021-08-03T19:45:17.172 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi163.front.sepia.ceph.com' reimaging 2021-08-03T19:45:17.172 INFO:teuthology.lock.ops:Updating [ubuntu@smithi163.front.sepia.ceph.com]: reset os type and version on server 2021-08-03T19:45:17.173 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2021-08-03T19:45:17.188 INFO:teuthology.lock.ops:Node 'ubuntu@smithi163.front.sepia.ceph.com' reimaging is complete 2021-08-03T19:45:17.367 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of ubuntu 20.04 2021-08-03T19:45:17.403 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of ubuntu 20.04 2021-08-03T19:45:17.938 INFO:teuthology.orchestra.console:Power off smithi163 2021-08-03T19:45:17.966 INFO:teuthology.orchestra.console:Power off smithi149 2021-08-03T19:45:26.329 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2021-08-03T19:45:26.430 INFO:teuthology.orchestra.console:Power on smithi149 2021-08-03T19:45:30.436 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2021-08-03T19:45:30.538 INFO:teuthology.orchestra.console:Power on smithi163 2021-08-03T19:45:34.795 INFO:teuthology.orchestra.console:Power on for smithi149 completed 2021-08-03T19:45:34.897 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish 2021-08-03T19:45:38.903 INFO:teuthology.orchestra.console:Power on for smithi163 completed 2021-08-03T19:45:39.004 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish 2021-08-03T19:51:09.628 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-03T19:51:10.066 INFO:teuthology.provision.fog.smithi149:Node is ready 2021-08-03T19:51:10.073 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com 2021-08-03T19:51:10.081 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149 2021-08-03T19:51:10.249 INFO:teuthology.provision.fog.smithi149:Deploy complete! 2021-08-03T19:51:25.332 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-03T19:51:25.760 INFO:teuthology.provision.fog.smithi163:Node is ready 2021-08-03T19:51:25.766 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163.front.sepia.ceph.com 2021-08-03T19:51:25.819 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163 2021-08-03T19:51:26.017 INFO:teuthology.provision.fog.smithi163:Deploy complete! 2021-08-03T19:51:26.204 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com 2021-08-03T19:51:26.205 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-03T19:51:26.225 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com 2021-08-03T19:51:26.226 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-03T19:51:26.241 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server 2021-08-03T19:51:26.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-03T19:51:26.423 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2021-08-03T19:51:26.430 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Ubuntu" 2021-08-03T19:51:26.430 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-08-03T19:51:26.430 INFO:teuthology.orchestra.run.smithi149.stdout:ID=ubuntu 2021-08-03T19:51:26.431 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE=debian 2021-08-03T19:51:26.431 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-08-03T19:51:26.431 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="20.04" 2021-08-03T19:51:26.431 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.ubuntu.com/" 2021-08-03T19:51:26.432 INFO:teuthology.orchestra.run.smithi149.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-08-03T19:51:26.432 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-08-03T19:51:26.432 INFO:teuthology.orchestra.run.smithi149.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-08-03T19:51:26.432 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_CODENAME=focal 2021-08-03T19:51:26.432 INFO:teuthology.orchestra.run.smithi149.stdout:UBUNTU_CODENAME=focal 2021-08-03T19:51:26.433 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2021-08-03T19:51:26.449 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server 2021-08-03T19:51:26.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-03T19:51:26.618 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2021-08-03T19:51:26.666 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Ubuntu" 2021-08-03T19:51:26.666 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-08-03T19:51:26.667 INFO:teuthology.orchestra.run.smithi163.stdout:ID=ubuntu 2021-08-03T19:51:26.667 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE=debian 2021-08-03T19:51:26.667 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-08-03T19:51:26.667 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="20.04" 2021-08-03T19:51:26.667 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.ubuntu.com/" 2021-08-03T19:51:26.668 INFO:teuthology.orchestra.run.smithi163.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-08-03T19:51:26.668 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-08-03T19:51:26.668 INFO:teuthology.orchestra.run.smithi163.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-08-03T19:51:26.668 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_CODENAME=focal 2021-08-03T19:51:26.669 INFO:teuthology.orchestra.run.smithi163.stdout:UBUNTU_CODENAME=focal 2021-08-03T19:51:26.669 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2021-08-03T19:51:26.751 INFO:teuthology.dispatcher.supervisor:Running job 6309429 2021-08-03T19:51:26.752 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2021-08-03_18:59:59-rados-wip-yuri-testing-2021-07-27-0830-pacific-distro-basic-smithi/6309429 --name nojha-2021-08-03_18:59:59-rados-wip-yuri-testing-2021-07-27-0830-pacific-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} -- /home/teuthworker/archive/nojha-2021-08-03_18:59:59-rados-wip-yuri-testing-2021-07-27-0830-pacific-distro-basic-smithi/6309429/orig.config.yaml 2021-08-03T19:51:26.755 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2021-08-03_18:59:59-rados-wip-yuri-testing-2021-07-27-0830-pacific-distro-basic-smithi/6309429 2021-08-03T19:51:26.755 INFO:teuthology.dispatcher.supervisor:Job PID: 21266 2021-08-03T19:51:26.756 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-08-04T07:52:13.611 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-08-04T07:52:13.834 INFO:teuthology.kill:Killing Pids: {21266} 2021-08-04T07:52:15.122 INFO:teuthology.kill:Nuking machines: ['smithi149', 'smithi163'] 2021-08-04T07:52:16.245 INFO:teuthology.kill:2021-08-04 07:52:16,244.244 INFO:teuthology.nuke:targets: 2021-08-04T07:52:16.246 INFO:teuthology.kill: smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCz1xQ/F4fp4w4d27U3yDNaELLaXlD+xCnBYBkD9SVcUI8prtto/JmP6ObybWhM1oiZWti8q+cYAsKWQv7nuMxKsrpIk2MWWedu+e6XRMLm/SRUsKkpqCNcrQq9wNekdprxc6EbwqoPlEMuB7o82LaW85zb6u3r61JXlnroEbjexk2N8kvyupSRQmPtaqASHhuncOqEvKXB2tpXDJWAyUBNn/uG8U1rkexD923O17zRKpr79HpnXnWXGxC1pZoJWHArKuyQ+dUgMowbxEfahwrIHEt08gTRLUK++CYlhwZ59JaB50vvVco3jr2ch1cL2YhXzS1ztQUw3WBjr5Wk+qSOC/Dh7JbtBsM/EtY8lIT5GamOGxzh63OOk4jMAuaKCMM/MGXhWSusVCqGVmAFEXqDT7YlWj0MFZFcabgDqzCp+PIzaMsrorhgaiDSyCKaccQ7v74IqmVK22ZEoACOhWcUFGPX3giUpUfsEJM1f1zjJfbiZRjzpZXH0A2kZdc7dKU= 2021-08-04T07:52:16.247 INFO:teuthology.kill: smithi163.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD1VLDC5d16kDAoqPx1etF7tS9ZcHIW6RBxbazWEhu5a70Btnk6YMKQVQaIj8CyfxfW21fr/iR+SbJ6DL9nTEkZWOZkXdLhHc91lk3wneIFnh0u80GYov0RBjtAC30VP+/Ic3HP9iACDSs5DFxu8AOCORts3zwFXjS+UXQRODm/LVuLOqaout9vK5qHrEqDKreaQ6C5CFjEyaCFJfGiF9DSCNYV6pUEPptqyWzBkk/MoVzea/9ddXQW7M3Yt0kGmCKr3LDrUTj7KWKDAhB96l+BT/5LeqnJmil81qH+SSm78i7iYXjSoiXc1oKbCH4hr63ci5DRBmaxxD9S6uapNMeo4c9ALUBtJp+IGbF9xPb86slZ1GL9tNeuRIKe4uNsUnw8MW0PyNI/9wHq13BtGGtrG33o0FqWBqA/dRsLuY37zQC31LiIonOlJgVvMjoc6e0p+50LGp0Mpv8LbVlfGETkgfaCarRQEgyAlv2tft9lHm38dggR0H/vvg5ST5vZnD0= 2021-08-04T07:52:16.247 INFO:teuthology.kill:2021-08-04 07:52:16,245.245 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-04T07:52:16.248 INFO:teuthology.kill:2021-08-04 07:52:16,247.247 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-04T07:52:16.316 INFO:teuthology.kill:2021-08-04 07:52:16,316.316 INFO:teuthology.orchestra.console:Power off smithi149 2021-08-04T07:52:16.337 INFO:teuthology.kill:2021-08-04 07:52:16,336.336 INFO:teuthology.orchestra.console:Power off smithi163 2021-08-04T07:52:24.701 INFO:teuthology.kill:2021-08-04 07:52:24,700.700 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2021-08-04T07:52:24.706 INFO:teuthology.kill:2021-08-04 07:52:24,706.706 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2021-08-04T07:52:24.956 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-08-04T07:52:24.956 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-08-04T07:52:24.957 INFO:teuthology.misc:Compressing logs... 2021-08-04T07:52:24.957 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-04T07:53:25.019 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-08-04T07:53:25.081 INFO:teuthology.kill:No teuthology processes running 2021-08-04T07:53:26.144 INFO:teuthology.kill:Nuking machines: ['smithi149', 'smithi163'] 2021-08-04T07:53:27.075 INFO:teuthology.kill:2021-08-04 07:53:27,075.075 INFO:teuthology.nuke:targets: 2021-08-04T07:53:27.076 INFO:teuthology.kill: smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCz1xQ/F4fp4w4d27U3yDNaELLaXlD+xCnBYBkD9SVcUI8prtto/JmP6ObybWhM1oiZWti8q+cYAsKWQv7nuMxKsrpIk2MWWedu+e6XRMLm/SRUsKkpqCNcrQq9wNekdprxc6EbwqoPlEMuB7o82LaW85zb6u3r61JXlnroEbjexk2N8kvyupSRQmPtaqASHhuncOqEvKXB2tpXDJWAyUBNn/uG8U1rkexD923O17zRKpr79HpnXnWXGxC1pZoJWHArKuyQ+dUgMowbxEfahwrIHEt08gTRLUK++CYlhwZ59JaB50vvVco3jr2ch1cL2YhXzS1ztQUw3WBjr5Wk+qSOC/Dh7JbtBsM/EtY8lIT5GamOGxzh63OOk4jMAuaKCMM/MGXhWSusVCqGVmAFEXqDT7YlWj0MFZFcabgDqzCp+PIzaMsrorhgaiDSyCKaccQ7v74IqmVK22ZEoACOhWcUFGPX3giUpUfsEJM1f1zjJfbiZRjzpZXH0A2kZdc7dKU= 2021-08-04T07:53:27.076 INFO:teuthology.kill: smithi163.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD1VLDC5d16kDAoqPx1etF7tS9ZcHIW6RBxbazWEhu5a70Btnk6YMKQVQaIj8CyfxfW21fr/iR+SbJ6DL9nTEkZWOZkXdLhHc91lk3wneIFnh0u80GYov0RBjtAC30VP+/Ic3HP9iACDSs5DFxu8AOCORts3zwFXjS+UXQRODm/LVuLOqaout9vK5qHrEqDKreaQ6C5CFjEyaCFJfGiF9DSCNYV6pUEPptqyWzBkk/MoVzea/9ddXQW7M3Yt0kGmCKr3LDrUTj7KWKDAhB96l+BT/5LeqnJmil81qH+SSm78i7iYXjSoiXc1oKbCH4hr63ci5DRBmaxxD9S6uapNMeo4c9ALUBtJp+IGbF9xPb86slZ1GL9tNeuRIKe4uNsUnw8MW0PyNI/9wHq13BtGGtrG33o0FqWBqA/dRsLuY37zQC31LiIonOlJgVvMjoc6e0p+50LGp0Mpv8LbVlfGETkgfaCarRQEgyAlv2tft9lHm38dggR0H/vvg5ST5vZnD0= 2021-08-04T07:53:27.096 INFO:teuthology.kill:2021-08-04 07:53:27,095.095 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-04T07:53:27.098 INFO:teuthology.kill:2021-08-04 07:53:27,098.098 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-04T07:53:27.154 INFO:teuthology.kill:2021-08-04 07:53:27,154.154 INFO:teuthology.orchestra.console:Power off smithi163 2021-08-04T07:53:27.188 INFO:teuthology.kill:2021-08-04 07:53:27,187.187 INFO:teuthology.orchestra.console:Power off smithi149 2021-08-04T07:53:27.295 INFO:teuthology.kill:2021-08-04 07:53:27,295.295 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2021-08-04T07:53:27.325 INFO:teuthology.kill:2021-08-04 07:53:27,325.325 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2021-08-04T07:53:27.436 INFO:teuthology.kill:2021-08-04 07:53:27,436.436 INFO:teuthology.lock.ops:unlocked smithi163.front.sepia.ceph.com 2021-08-04T07:53:27.463 INFO:teuthology.kill:2021-08-04 07:53:27,463.463 INFO:teuthology.lock.ops:unlocked smithi149.front.sepia.ceph.com 2021-08-04T07:55:27.759 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-08-04T07:55:27.852 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-08-04T07:55:27.853 INFO:teuthology.nuke:Checking targets against current locks 2021-08-04T07:55:27.966 INFO:teuthology.nuke:Not nuking smithi149.front.sepia.ceph.com because description doesn't match 2021-08-04T07:55:27.967 INFO:teuthology.nuke:Not nuking smithi163.front.sepia.ceph.com because description doesn't match