2023-03-13T17:25:33.643 INFO:root:teuthology version: 0.0.1.dev57+g5c9057f 2023-03-13T17:25:33.876 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging 2023-03-13T17:25:33.877 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server 2023-03-13T17:25:33.877 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-03-13T17:25:33.903 INFO:teuthology.lock.ops:Node 'smithi072.front.sepia.ceph.com' reimaging is complete 2023-03-13T17:25:33.903 INFO:teuthology.lock.ops:Start node 'smithi123.front.sepia.ceph.com' reimaging 2023-03-13T17:25:33.903 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: reset os type and version on server 2023-03-13T17:25:33.904 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-03-13T17:25:33.927 INFO:teuthology.lock.ops:Node 'smithi123.front.sepia.ceph.com' reimaging is complete 2023-03-13T17:25:33.928 INFO:teuthology.lock.ops:Start node 'smithi144.front.sepia.ceph.com' reimaging 2023-03-13T17:25:33.928 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: reset os type and version on server 2023-03-13T17:25:33.928 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-03-13T17:25:33.952 INFO:teuthology.lock.ops:Node 'smithi144.front.sepia.ceph.com' reimaging is complete 2023-03-13T17:25:33.953 INFO:teuthology.lock.ops:Start node 'smithi178.front.sepia.ceph.com' reimaging 2023-03-13T17:25:33.953 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: reset os type and version on server 2023-03-13T17:25:33.953 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-03-13T17:25:33.978 INFO:teuthology.lock.ops:Node 'smithi178.front.sepia.ceph.com' reimaging is complete 2023-03-13T17:25:34.062 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of ubuntu 20.04 2023-03-13T17:25:34.101 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of ubuntu 20.04 2023-03-13T17:25:34.129 INFO:teuthology.provision.fog.smithi144:Scheduling deploy of ubuntu 20.04 2023-03-13T17:25:34.167 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of ubuntu 20.04 2023-03-13T17:25:34.285 INFO:teuthology.orchestra.console:Power off smithi072 2023-03-13T17:25:34.336 INFO:teuthology.orchestra.console:Power off smithi123 2023-03-13T17:25:34.352 INFO:teuthology.orchestra.console:Power off smithi144 2023-03-13T17:25:34.407 INFO:teuthology.orchestra.console:Power off smithi178 2023-03-13T17:25:34.463 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2023-03-13T17:25:34.490 INFO:teuthology.orchestra.console:Power off for smithi123 completed 2023-03-13T17:25:34.519 INFO:teuthology.orchestra.console:Power off for smithi144 completed 2023-03-13T17:25:34.552 INFO:teuthology.orchestra.console:Power off for smithi178 completed 2023-03-13T17:25:34.565 INFO:teuthology.orchestra.console:Power on smithi072 2023-03-13T17:25:34.597 INFO:teuthology.orchestra.console:Power on smithi123 2023-03-13T17:25:34.627 INFO:teuthology.orchestra.console:Power on smithi144 2023-03-13T17:25:34.657 INFO:teuthology.orchestra.console:Power on smithi178 2023-03-13T17:25:38.854 INFO:teuthology.orchestra.console:Power on for smithi123 completed 2023-03-13T17:25:38.955 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish 2023-03-13T17:25:42.936 INFO:teuthology.orchestra.console:Power on for smithi072 completed 2023-03-13T17:25:42.993 INFO:teuthology.orchestra.console:Power on for smithi144 completed 2023-03-13T17:25:43.022 INFO:teuthology.orchestra.console:Power on for smithi178 completed 2023-03-13T17:25:43.037 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2023-03-13T17:25:43.094 INFO:teuthology.provision.fog.smithi144:Waiting for deploy to finish 2023-03-13T17:25:43.123 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish 2023-03-13T17:29:28.101 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-03-13T17:29:32.101 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-03-13T17:29:37.157 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-03-13T17:29:41.157 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-03-13T17:29:46.213 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-03-13T17:29:47.173 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:29:50.213 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-03-13T17:29:55.269 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-03-13T17:29:56.233 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:29:59.269 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-03-13T17:30:04.325 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-03-13T17:30:05.289 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:30:08.325 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-03-13T17:30:13.381 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-03-13T17:30:14.341 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:30:23.397 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:30:32.453 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:30:41.509 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:30:51.133 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-03-13T17:31:05.861 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:31:11.913 INFO:teuthology.provision.fog.smithi123:Node is ready 2023-03-13T17:31:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com 2023-03-13T17:31:11.926 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123 2023-03-13T17:31:12.044 INFO:teuthology.provision.fog.smithi123:Deploy complete! 2023-03-13T17:31:14.330 WARNING:teuthology.provision.fog:timed out 2023-03-13T17:31:14.917 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-03-13T17:31:17.477 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-03-13T17:31:20.536 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-03-13T17:31:21.033 INFO:teuthology.provision.fog.smithi144:Node is ready 2023-03-13T17:31:21.040 INFO:teuthology.orchestra.run.smithi144.stdout:smithi144.front.sepia.ceph.com 2023-03-13T17:31:21.048 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144 smithi144.front.sepia.ceph.com smithi144 2023-03-13T17:31:21.269 INFO:teuthology.provision.fog.smithi144:Deploy complete! 2023-03-13T17:31:22.145 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-03-13T17:31:26.533 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-03-13T17:31:35.589 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-03-13T17:31:43.094 INFO:teuthology.provision.fog.smithi178:Node is ready 2023-03-13T17:31:43.101 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com 2023-03-13T17:31:43.109 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178 2023-03-13T17:31:43.280 INFO:teuthology.provision.fog.smithi178:Deploy complete! 2023-03-13T17:31:44.645 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-03-13T17:31:53.701 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-03-13T17:32:59.705 WARNING:teuthology.provision.fog:timed out 2023-03-13T17:33:05.910 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-03-13T17:33:06.403 INFO:teuthology.provision.fog.smithi072:Node is ready 2023-03-13T17:33:06.409 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072.front.sepia.ceph.com 2023-03-13T17:33:06.416 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72 smithi072.front.sepia.ceph.com smithi072 2023-03-13T17:33:06.656 INFO:teuthology.provision.fog.smithi072:Deploy complete! 2023-03-13T17:33:07.062 INFO:teuthology.lock.ops:Checking smithi072.front.sepia.ceph.com 2023-03-13T17:33:07.063 INFO:teuthology.lock.ops:New key found. Updating... 2023-03-13T17:33:07.089 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com 2023-03-13T17:33:07.089 INFO:teuthology.lock.ops:New key found. Updating... 2023-03-13T17:33:07.105 INFO:teuthology.lock.ops:Checking smithi144.front.sepia.ceph.com 2023-03-13T17:33:07.106 INFO:teuthology.lock.ops:New key found. Updating... 2023-03-13T17:33:07.130 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com 2023-03-13T17:33:07.131 INFO:teuthology.lock.ops:New key found. Updating... 2023-03-13T17:33:07.158 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: set os type and version on server 2023-03-13T17:33:07.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-13T17:33:07.332 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2023-03-13T17:33:07.381 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Ubuntu" 2023-03-13T17:33:07.382 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-03-13T17:33:07.382 INFO:teuthology.orchestra.run.smithi072.stdout:ID=ubuntu 2023-03-13T17:33:07.382 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE=debian 2023-03-13T17:33:07.382 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-03-13T17:33:07.382 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="20.04" 2023-03-13T17:33:07.382 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.ubuntu.com/" 2023-03-13T17:33:07.383 INFO:teuthology.orchestra.run.smithi072.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-03-13T17:33:07.383 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-03-13T17:33:07.383 INFO:teuthology.orchestra.run.smithi072.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-03-13T17:33:07.383 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_CODENAME=focal 2023-03-13T17:33:07.383 INFO:teuthology.orchestra.run.smithi072.stdout:UBUNTU_CODENAME=focal 2023-03-13T17:33:07.384 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-03-13T17:33:07.414 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server 2023-03-13T17:33:07.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-03-13T17:33:07.671 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-03-13T17:33:07.677 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Ubuntu" 2023-03-13T17:33:07.678 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-03-13T17:33:07.678 INFO:teuthology.orchestra.run.smithi123.stdout:ID=ubuntu 2023-03-13T17:33:07.678 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE=debian 2023-03-13T17:33:07.678 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-03-13T17:33:07.678 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="20.04" 2023-03-13T17:33:07.678 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.ubuntu.com/" 2023-03-13T17:33:07.679 INFO:teuthology.orchestra.run.smithi123.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-03-13T17:33:07.679 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-03-13T17:33:07.679 INFO:teuthology.orchestra.run.smithi123.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-03-13T17:33:07.679 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_CODENAME=focal 2023-03-13T17:33:07.679 INFO:teuthology.orchestra.run.smithi123.stdout:UBUNTU_CODENAME=focal 2023-03-13T17:33:07.679 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-03-13T17:33:07.706 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: set os type and version on server 2023-03-13T17:33:07.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-03-13T17:33:07.959 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2023-03-13T17:33:07.965 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Ubuntu" 2023-03-13T17:33:07.966 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-03-13T17:33:07.966 INFO:teuthology.orchestra.run.smithi144.stdout:ID=ubuntu 2023-03-13T17:33:07.966 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE=debian 2023-03-13T17:33:07.966 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-03-13T17:33:07.966 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="20.04" 2023-03-13T17:33:07.966 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.ubuntu.com/" 2023-03-13T17:33:07.966 INFO:teuthology.orchestra.run.smithi144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-03-13T17:33:07.967 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-03-13T17:33:07.967 INFO:teuthology.orchestra.run.smithi144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-03-13T17:33:07.967 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_CODENAME=focal 2023-03-13T17:33:07.967 INFO:teuthology.orchestra.run.smithi144.stdout:UBUNTU_CODENAME=focal 2023-03-13T17:33:07.967 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-03-13T17:33:07.989 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server 2023-03-13T17:33:07.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-03-13T17:33:08.164 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2023-03-13T17:33:08.170 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Ubuntu" 2023-03-13T17:33:08.170 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-03-13T17:33:08.170 INFO:teuthology.orchestra.run.smithi178.stdout:ID=ubuntu 2023-03-13T17:33:08.170 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE=debian 2023-03-13T17:33:08.171 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-03-13T17:33:08.171 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="20.04" 2023-03-13T17:33:08.171 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.ubuntu.com/" 2023-03-13T17:33:08.171 INFO:teuthology.orchestra.run.smithi178.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-03-13T17:33:08.171 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-03-13T17:33:08.171 INFO:teuthology.orchestra.run.smithi178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-03-13T17:33:08.171 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_CODENAME=focal 2023-03-13T17:33:08.172 INFO:teuthology.orchestra.run.smithi178.stdout:UBUNTU_CODENAME=focal 2023-03-13T17:33:08.172 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-03-13T17:33:08.286 INFO:teuthology.dispatcher.supervisor:Running job 7204520 2023-03-13T17:33:08.287 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2023-03-12_18:26:24-rados-WIP_GBH_snap_mapper_2-distro-default-smithi/7204520 --name benhanokh-2023-03-12_18:26:24-rados-WIP_GBH_snap_mapper_2-distro-default-smithi --description rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} -- /home/teuthworker/archive/benhanokh-2023-03-12_18:26:24-rados-WIP_GBH_snap_mapper_2-distro-default-smithi/7204520/orig.config.yaml 2023-03-13T17:33:08.291 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2023-03-12_18:26:24-rados-WIP_GBH_snap_mapper_2-distro-default-smithi/7204520 2023-03-13T17:33:08.291 INFO:teuthology.dispatcher.supervisor:Job PID: 22186 2023-03-13T17:33:08.292 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-03-14T05:34:06.117 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-03-14T05:34:06.275 INFO:teuthology.kill:Killing Pids: {22186} 2023-03-14T05:34:06.505 ERROR:teuthology.dispatcher.supervisor:Failed to kill job Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog kill_job(job_info['name'], job_info['job_id'], File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"]) KeyError: 'status' 2023-03-14T05:34:06.541 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2023-03-14T05:34:06.542 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2023-03-14T05:34:06.542 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2023-03-14T05:34:06.542 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2023-03-14T05:34:06.543 INFO:teuthology.misc:Compressing logs... 2023-03-14T05:34:06.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-14T05:34:06.721 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-03-14T05:34:06.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-03-14T05:34:07.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-03-14T05:34:07.734 INFO:teuthology.misc:Compressing logs... 2023-03-14T05:36:08.178 INFO:teuthology.misc:Compressing logs... 2023-03-14T05:36:11.844 INFO:teuthology.kill:No teuthology processes running 2023-03-14T05:36:11.844 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog kill_job(job_info['name'], job_info['job_id'], File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"]) File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__ self.metric = Counter( File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__ registry.register(self) File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register raise ValueError( ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'} 2023-03-14T05:38:12.123 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-03-14T05:38:12.303 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-03-14T05:38:12.307 INFO:teuthology.nuke:Checking targets against current locks 2023-03-14T05:38:12.360 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-14T05:38:12.387 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-14T05:38:12.413 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-14T05:38:12.441 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-14T05:38:12.451 INFO:teuthology.orchestra.console:Power off smithi072 2023-03-14T05:38:12.498 INFO:teuthology.orchestra.console:Power off smithi144 2023-03-14T05:38:12.515 INFO:teuthology.orchestra.console:Power off smithi123 2023-03-14T05:38:12.594 INFO:teuthology.orchestra.console:Power off smithi178 2023-03-14T05:38:20.942 INFO:teuthology.orchestra.console:Power off for smithi144 completed 2023-03-14T05:38:21.103 INFO:teuthology.lock.ops:unlocked: smithi144.front.sepia.ceph.com 2023-03-14T05:38:25.048 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2023-03-14T05:38:25.062 INFO:teuthology.orchestra.console:Power off for smithi123 completed 2023-03-14T05:38:25.073 INFO:teuthology.orchestra.console:Power off for smithi178 completed 2023-03-14T05:38:25.205 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com 2023-03-14T05:38:25.227 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com 2023-03-14T05:38:25.234 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com