2023-03-17T13:13:31.469 INFO:root:teuthology version: 0.0.1.dev57+g5c9057f
2023-03-17T13:13:31.911 INFO:teuthology.lock.ops:Start node 'smithi008.front.sepia.ceph.com' reimaging
2023-03-17T13:13:32.261 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: reset os type and version on server
2023-03-17T13:13:32.261 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server
2023-03-17T13:13:32.281 INFO:teuthology.lock.ops:Node 'smithi008.front.sepia.ceph.com' reimaging is complete
2023-03-17T13:13:32.948 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging
2023-03-17T13:13:34.045 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server
2023-03-17T13:13:34.045 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2023-03-17T13:13:34.072 INFO:teuthology.lock.ops:Node 'smithi098.front.sepia.ceph.com' reimaging is complete
2023-03-17T13:13:35.889 INFO:teuthology.lock.ops:Start node 'smithi145.front.sepia.ceph.com' reimaging
2023-03-17T13:13:35.889 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: reset os type and version on server
2023-03-17T13:13:35.889 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server
2023-03-17T13:13:35.914 INFO:teuthology.lock.ops:Node 'smithi145.front.sepia.ceph.com' reimaging is complete
2023-03-17T13:13:36.411 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging
2023-03-17T13:13:36.411 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server
2023-03-17T13:13:36.411 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2023-03-17T13:13:36.438 INFO:teuthology.lock.ops:Node 'smithi146.front.sepia.ceph.com' reimaging is complete
2023-03-17T13:13:37.151 INFO:teuthology.lock.ops:Start node 'smithi158.front.sepia.ceph.com' reimaging
2023-03-17T13:13:37.151 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: reset os type and version on server
2023-03-17T13:13:37.151 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2023-03-17T13:13:37.180 INFO:teuthology.lock.ops:Node 'smithi158.front.sepia.ceph.com' reimaging is complete
2023-03-17T13:13:37.611 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of centos 8.stream
2023-03-17T13:13:38.277 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 8.stream
2023-03-17T13:13:38.759 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of centos 8.stream
2023-03-17T13:13:38.820 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 8.stream
2023-03-17T13:13:39.716 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of centos 8.stream
2023-03-17T13:13:40.027 INFO:teuthology.orchestra.console:Power off smithi008
2023-03-17T13:13:40.239 INFO:teuthology.orchestra.console:Power off smithi145
2023-03-17T13:13:41.305 INFO:teuthology.orchestra.console:Power off smithi098
2023-03-17T13:13:42.194 INFO:teuthology.orchestra.console:Power off smithi146
2023-03-17T13:13:42.255 INFO:teuthology.orchestra.console:Power off for smithi145 completed
2023-03-17T13:13:42.864 INFO:teuthology.orchestra.console:Power off for smithi098 completed
2023-03-17T13:13:42.964 INFO:teuthology.orchestra.console:Power on smithi145
2023-03-17T13:13:43.661 INFO:teuthology.orchestra.console:Power on smithi098
2023-03-17T13:13:43.675 INFO:teuthology.orchestra.console:Power off smithi158
2023-03-17T13:13:44.298 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2023-03-17T13:13:44.880 INFO:teuthology.orchestra.console:Power on smithi146
2023-03-17T13:13:45.334 INFO:teuthology.orchestra.console:Power off for smithi158 completed
2023-03-17T13:13:45.785 INFO:teuthology.orchestra.console:Power on smithi158
2023-03-17T13:13:49.027 INFO:teuthology.orchestra.console:Power on for smithi098 completed
2023-03-17T13:13:49.404 INFO:teuthology.orchestra.console:Power on for smithi145 completed
2023-03-17T13:13:49.504 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish
2023-03-17T13:13:49.820 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish
2023-03-17T13:13:50.033 INFO:teuthology.orchestra.console:Power off for smithi008 completed
2023-03-17T13:13:50.716 INFO:teuthology.orchestra.console:Power on for smithi146 completed
2023-03-17T13:13:50.816 INFO:teuthology.orchestra.console:Power on smithi008
2023-03-17T13:13:51.941 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2023-03-17T13:13:52.576 INFO:teuthology.orchestra.console:Power on for smithi158 completed
2023-03-17T13:13:53.785 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish
2023-03-17T13:13:56.711 INFO:teuthology.orchestra.console:Power on for smithi008 completed
2023-03-17T13:13:57.697 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish
2023-03-17T13:16:38.821 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-17T13:16:40.279 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-03-17T13:16:47.077 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.8
2023-03-17T13:16:49.353 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-17T13:16:51.339 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-03-17T13:16:56.421 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-03-17T13:16:57.957 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.8
2023-03-17T13:16:58.469 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-03-17T13:17:00.421 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-03-17T13:17:01.164 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-17T13:17:05.669 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-03-17T13:17:07.173 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.8
2023-03-17T13:17:09.897 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-03-17T13:17:11.429 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-03-17T13:17:11.429 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-17T13:17:15.913 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-03-17T13:17:18.981 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.8
2023-03-17T13:17:21.509 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-03-17T13:17:23.699 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-17T13:17:23.700 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-03-17T13:17:29.125 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-03-17T13:17:30.341 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.8
2023-03-17T13:17:32.901 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-17T13:17:33.577 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-03-17T13:17:35.141 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-03-17T13:17:38.917 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-03-17T13:17:43.877 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-03-17T13:17:49.861 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-03-17T13:17:55.205 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-03-17T13:17:59.685 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-03-17T13:18:05.065 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-03-17T13:18:08.899 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-17T13:18:09.463 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-03-17T13:18:12.565 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-17T13:18:14.185 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-03-17T13:18:16.811 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-17T13:18:24.881 INFO:teuthology.provision.fog.smithi008:Node is ready
2023-03-17T13:18:25.922 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com
2023-03-17T13:18:28.190 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008
2023-03-17T13:18:30.368 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-17T13:18:32.234 INFO:teuthology.provision.fog.smithi008:Deploy complete!
2023-03-17T13:18:35.804 INFO:teuthology.provision.fog.smithi145:Node is ready
2023-03-17T13:18:35.827 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com
2023-03-17T13:18:38.066 INFO:teuthology.provision.fog.smithi098:Node is ready
2023-03-17T13:18:38.083 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145
2023-03-17T13:18:38.488 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com
2023-03-17T13:18:38.549 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098
2023-03-17T13:18:40.873 INFO:teuthology.provision.fog.smithi145:Deploy complete!
2023-03-17T13:18:41.124 INFO:teuthology.provision.fog.smithi098:Deploy complete!
2023-03-17T13:18:52.492 INFO:teuthology.provision.fog.smithi146:Node is ready
2023-03-17T13:18:53.568 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2023-03-17T13:18:54.763 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2023-03-17T13:18:56.056 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-17T13:18:56.939 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2023-03-17T13:19:16.354 INFO:teuthology.provision.fog.smithi158:Node is ready
2023-03-17T13:19:17.378 INFO:teuthology.orchestra.run.smithi158.stdout:smithi158.front.sepia.ceph.com
2023-03-17T13:19:17.896 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.158 smithi158.front.sepia.ceph.com smithi158
2023-03-17T13:19:24.047 INFO:teuthology.provision.fog.smithi158:Deploy complete!
2023-03-17T13:19:25.520 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com
2023-03-17T13:19:27.201 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-17T13:19:27.223 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com
2023-03-17T13:19:27.990 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-17T13:19:28.015 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com
2023-03-17T13:19:28.804 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-17T13:19:28.823 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2023-03-17T13:19:30.978 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-17T13:19:31.006 INFO:teuthology.lock.ops:Checking smithi158.front.sepia.ceph.com
2023-03-17T13:19:31.762 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-17T13:19:31.791 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server
2023-03-17T13:19:32.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-17T13:19:33.126 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64
2023-03-17T13:19:34.465 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream"
2023-03-17T13:19:35.718 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8"
2023-03-17T13:19:35.718 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos"
2023-03-17T13:19:35.718 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora"
2023-03-17T13:19:35.718 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8"
2023-03-17T13:19:35.718 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8"
2023-03-17T13:19:35.718 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-17T13:19:35.718 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31"
2023-03-17T13:19:35.719 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-17T13:19:35.719 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/"
2023-03-17T13:19:35.719 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-17T13:19:35.719 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-17T13:19:35.719 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-17T13:19:35.721 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server
2023-03-17T13:19:35.756 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server
2023-03-17T13:19:36.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-17T13:19:37.271 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64
2023-03-17T13:19:38.077 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream"
2023-03-17T13:19:38.934 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8"
2023-03-17T13:19:38.934 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos"
2023-03-17T13:19:38.934 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora"
2023-03-17T13:19:38.934 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8"
2023-03-17T13:19:38.934 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8"
2023-03-17T13:19:38.934 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-17T13:19:38.934 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2023-03-17T13:19:38.935 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-17T13:19:38.935 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/"
2023-03-17T13:19:38.935 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-17T13:19:38.935 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-17T13:19:38.935 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-17T13:19:38.935 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2023-03-17T13:19:38.955 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server
2023-03-17T13:19:39.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-17T13:19:39.657 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64
2023-03-17T13:19:40.319 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream"
2023-03-17T13:19:40.745 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8"
2023-03-17T13:19:40.746 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos"
2023-03-17T13:19:40.746 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora"
2023-03-17T13:19:40.746 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8"
2023-03-17T13:19:40.746 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8"
2023-03-17T13:19:40.746 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-17T13:19:40.746 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31"
2023-03-17T13:19:40.747 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-17T13:19:40.747 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/"
2023-03-17T13:19:40.747 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-17T13:19:40.747 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-17T13:19:40.747 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-17T13:19:40.747 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server
2023-03-17T13:19:40.780 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2023-03-17T13:19:41.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-17T13:19:41.794 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2023-03-17T13:19:41.992 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-17T13:19:42.762 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-17T13:19:42.763 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2023-03-17T13:19:42.797 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: set os type and version on server
2023-03-17T13:19:43.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-17T13:19:43.468 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64
2023-03-17T13:19:44.161 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream"
2023-03-17T13:19:44.541 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/"
2023-03-17T13:19:44.542 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-17T13:19:44.543 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-17T13:19:44.543 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-17T13:19:44.543 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2023-03-17T13:19:44.647 INFO:teuthology.dispatcher.supervisor:Running job 7209178
2023-03-17T13:19:45.474 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-03-15_23:29:37-rados-wip-yuri2-testing-2023-03-15-1204-quincy-distro-default-smithi/7209178 --name yuriw-2023-03-15_23:29:37-rados-wip-yuri2-testing-2023-03-15-1204-quincy-distro-default-smithi --description rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_orch_cli_mon} -- /home/teuthworker/archive/yuriw-2023-03-15_23:29:37-rados-wip-yuri2-testing-2023-03-15-1204-quincy-distro-default-smithi/7209178/orig.config.yaml
2023-03-17T13:19:45.478 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-03-15_23:29:37-rados-wip-yuri2-testing-2023-03-15-1204-quincy-distro-default-smithi/7209178
2023-03-17T13:19:46.379 INFO:teuthology.dispatcher.supervisor:Job PID: 11849
2023-03-17T13:19:46.379 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-18T01:20:39.330 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:20:42.972 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:20:43.469 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
KeyError: 'status'
2023-03-18T01:20:44.115 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:20:44.846 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:20:44.846 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:20:45.197 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:20:45.197 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:20:45.198 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:20:45.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:20:45.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:20:46.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:20:48.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:20:49.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:20:51.486 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:20:53.869 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-18T01:20:55.217 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-18T01:20:55.217 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-18T01:20:55.963 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:21:05.044 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-mon.smithi158.log: file size changed while zipping
2023-03-18T01:21:06.219 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2023-03-18T01:21:07.387 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.9.log: file size changed while zipping
2023-03-18T01:21:13.316 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.14.log: file size changed while zipping
2023-03-18T01:21:14.868 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-mon.smithi146.log: file size changed while zipping
2023-03-18T01:21:14.868 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.8.log: file size changed while zipping
2023-03-18T01:21:19.944 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.12.log: file size changed while zipping
2023-03-18T01:21:22.537 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.11.log: file size changed while zipping
2023-03-18T01:21:23.687 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2023-03-18T01:21:28.739 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.13.log: file size changed while zipping
2023-03-18T01:21:30.277 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3d1bf790-c4ca-11ed-9afc-001a4aab830c/ceph-osd.10.log: file size changed while zipping
2023-03-18T01:21:50.491 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:21:51.849 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:23:53.845 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:23:54.745 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:23:56.610 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:23:58.345 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:23:59.266 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:23:59.266 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:23:59.266 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:23:59.267 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:23:59.267 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:23:59.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:23:59.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:24:01.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:24:02.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:24:04.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:24:07.272 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:24:11.024 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:24:33.763 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:24:34.975 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:26:36.414 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:26:37.555 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:26:39.900 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:26:41.945 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:26:43.860 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:26:43.860 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:26:43.860 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:26:43.860 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:26:43.860 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:26:43.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:26:44.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:26:45.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:26:49.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:26:52.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:26:53.972 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:26:55.467 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:27:16.960 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:27:19.873 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:29:23.689 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:29:25.886 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:29:27.426 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:29:29.283 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:29:30.839 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:29:30.839 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:29:30.839 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:29:30.839 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:29:30.839 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:29:30.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:29:31.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:29:33.290 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:29:34.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:29:36.906 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:29:38.669 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:29:40.301 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:30:02.037 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:30:04.993 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:32:07.060 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:32:09.173 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:32:11.647 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:32:13.340 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:32:14.653 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:32:14.653 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:32:14.653 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:32:14.653 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:32:14.653 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:32:14.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:32:14.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:32:17.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:32:20.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:32:22.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:32:23.583 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:32:25.796 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:32:47.044 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:32:50.492 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:34:52.921 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:34:54.403 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:34:56.416 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:34:57.786 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:34:59.401 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:35:00.426 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:35:00.426 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:35:00.426 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:35:00.426 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:35:00.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:35:00.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:35:02.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:35:03.041 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:35:04.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:35:05.019 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:35:06.835 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:35:27.360 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:35:31.305 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:37:31.974 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:37:32.964 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:37:35.341 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:37:37.531 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:37:38.586 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:37:38.586 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:37:40.193 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:37:40.193 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:37:40.193 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:37:40.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:37:40.371 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:37:42.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:37:43.726 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:37:46.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:37:49.234 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:37:51.392 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:38:21.414 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:38:22.553 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:40:23.492 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:40:28.329 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:40:30.140 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:40:31.836 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:40:34.251 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:40:34.251 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:40:34.251 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:40:34.251 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:40:34.251 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:40:34.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:40:34.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:40:35.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:40:36.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:40:38.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:40:40.266 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:40:42.291 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:41:03.332 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:41:04.015 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:43:05.253 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:43:05.946 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:43:06.876 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:43:07.645 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:43:08.877 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:43:08.877 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:43:08.877 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:43:08.877 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:43:08.877 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:43:08.877 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:43:09.068 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:43:09.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:43:10.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:43:10.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:43:11.935 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:43:12.996 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:43:33.281 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:43:33.927 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:45:35.941 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:45:37.684 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:45:38.910 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:45:40.021 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:45:41.009 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:45:41.978 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:45:41.978 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:45:41.978 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:45:41.978 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:45:41.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:45:42.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:45:44.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:45:45.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:45:48.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:45:50.221 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:45:54.920 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:46:17.216 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:46:21.237 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:48:22.897 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:48:24.111 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:48:26.122 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:48:28.395 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:48:31.529 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:48:31.529 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:48:31.529 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:48:31.529 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:48:31.530 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:48:31.530 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:48:32.716 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:48:33.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:48:33.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:48:34.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:48:36.369 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:48:38.173 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:48:58.990 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:49:02.894 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:51:04.476 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:51:06.278 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:51:08.479 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:51:09.986 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:51:11.228 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:51:11.228 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:51:11.228 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:51:11.228 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:51:11.229 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:51:11.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:51:11.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:51:12.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:51:13.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:51:15.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:51:19.039 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:51:21.711 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:51:43.020 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:51:44.134 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:53:45.933 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:53:48.352 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:53:52.273 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:53:54.227 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:53:59.712 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:53:59.712 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:53:59.712 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:53:59.712 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:53:59.712 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:53:59.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:53:59.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:54:01.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:54:03.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:54:05.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:54:11.964 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:54:14.545 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:54:37.995 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:54:41.763 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:56:44.148 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:56:49.121 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:56:50.558 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:56:53.105 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:56:54.446 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:56:54.447 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:56:54.447 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:56:54.447 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:56:54.447 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:56:54.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:56:54.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:56:56.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:56:57.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:56:58.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:57:00.345 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:57:02.432 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:57:23.809 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:57:24.572 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:59:25.274 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T01:59:26.233 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T01:59:26.967 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T01:59:28.465 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T01:59:31.312 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T01:59:33.161 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T01:59:33.161 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T01:59:33.161 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T01:59:33.161 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:59:33.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T01:59:33.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T01:59:38.140 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T01:59:39.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T01:59:40.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T01:59:42.733 INFO:teuthology.misc:Compressing logs...
2023-03-18T01:59:46.580 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:00:08.945 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:00:09.648 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:02:11.109 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:02:11.860 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:02:12.445 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:02:13.417 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:02:13.770 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:02:13.770 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:02:13.770 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:02:14.752 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:02:14.752 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:02:14.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:02:14.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:02:16.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:02:17.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:02:18.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:02:19.514 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:02:21.109 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:02:42.018 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:02:45.656 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:04:48.884 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:04:50.104 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:04:52.602 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:04:55.250 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:04:57.234 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:04:57.234 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:04:57.235 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:04:57.235 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:04:57.235 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:04:57.235 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:04:57.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:04:59.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:05:03.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:05:05.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:05:07.526 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:05:08.540 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:05:29.441 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:05:33.265 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:07:36.097 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:07:36.597 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:07:38.402 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:07:39.402 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:07:43.431 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:07:44.673 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:07:44.674 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:07:44.674 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:07:44.674 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:07:44.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:07:44.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:07:46.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:07:48.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:07:52.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:07:54.813 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:07:59.135 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:08:19.581 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:08:20.826 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:10:22.330 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:10:23.852 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:10:25.253 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:10:28.803 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:10:30.190 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:10:32.120 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:10:32.121 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:10:32.121 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:10:32.121 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:10:32.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:10:33.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:10:38.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:10:40.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:10:41.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:10:43.219 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:10:45.113 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:11:06.184 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:11:07.479 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:13:09.455 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:13:10.431 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:13:11.218 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:13:12.046 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:13:13.565 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:13:13.566 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:13:14.014 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:13:14.014 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:13:14.014 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:13:14.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:13:14.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:13:16.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:13:17.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:13:18.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:13:20.005 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:13:21.512 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:13:42.945 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:13:45.487 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:15:46.942 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:15:48.989 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:15:51.372 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:15:52.680 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:15:53.689 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:15:54.840 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:15:54.841 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:15:54.841 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:15:54.841 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:15:55.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:15:56.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:15:58.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:15:59.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:16:00.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:16:01.816 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:16:03.889 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:16:25.411 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:16:25.991 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:18:26.734 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:18:28.708 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:18:30.627 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:18:31.668 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:18:33.488 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:18:33.488 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:18:33.488 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:18:33.488 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:18:33.488 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:18:33.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:18:33.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:18:35.501 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:18:36.332 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:18:40.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:18:43.555 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:18:47.192 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:19:11.445 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:19:12.285 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:21:17.195 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:21:19.113 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:21:20.292 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:21:21.449 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:21:22.452 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:21:22.452 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:21:22.452 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:21:22.452 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:21:22.452 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:21:22.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:21:22.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:21:23.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:21:25.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:21:25.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:21:27.456 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:21:28.679 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:21:49.550 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:21:50.640 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:23:51.485 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:23:52.892 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:23:55.003 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:23:57.418 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:24:00.022 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:24:00.023 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:24:00.023 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:24:00.023 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:24:00.023 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:24:00.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:24:00.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:24:01.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:24:01.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:24:02.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:24:03.869 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:24:05.112 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:24:25.506 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:24:26.770 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:26:28.961 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:26:30.575 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:26:31.756 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:26:32.863 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:26:33.962 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:26:33.962 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:26:33.962 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:26:33.962 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:26:33.962 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:26:33.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:26:36.180 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:26:37.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:26:39.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:26:43.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:26:45.607 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:26:52.170 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:27:13.264 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:27:22.181 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:29:32.081 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:29:33.252 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:29:34.369 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:29:35.134 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:29:37.324 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:29:38.421 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:29:39.262 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:29:39.262 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:29:39.262 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:29:40.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:29:40.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:29:41.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:29:42.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:29:42.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:29:45.027 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:29:47.072 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:30:08.687 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:30:10.105 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:32:11.701 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:32:12.555 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:32:13.980 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:32:15.656 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:32:22.119 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:32:22.119 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:32:22.120 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:32:22.120 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:32:22.120 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:32:22.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:32:22.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:32:25.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:32:29.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:32:31.709 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:32:33.163 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:32:37.293 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:32:59.475 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:33:00.707 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:35:02.034 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:35:02.292 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:35:03.666 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:35:04.118 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:35:05.433 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:35:05.433 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:35:05.433 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:35:05.434 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:35:05.434 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:35:05.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:35:05.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:35:06.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:35:08.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:35:09.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:35:11.598 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:35:14.964 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:35:36.578 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:35:37.300 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:37:39.850 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:37:41.235 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:37:43.438 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:37:44.788 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:37:46.192 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:37:46.192 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:37:46.192 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:37:46.192 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:37:46.192 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:37:46.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:37:46.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:37:47.753 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:37:48.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:37:51.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:37:54.061 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:37:59.433 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:38:22.318 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:38:22.926 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:40:24.589 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:40:25.763 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:40:27.957 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:40:28.931 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:40:30.077 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:40:30.077 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:40:30.078 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:40:30.078 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:40:30.078 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:40:30.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:40:31.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:40:32.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:40:33.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:40:34.116 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:40:35.300 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:40:36.910 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:40:57.508 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:40:58.405 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:43:00.115 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:43:01.325 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:43:04.381 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:43:06.033 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:43:09.732 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:43:09.732 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:43:09.732 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:43:09.732 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:43:09.732 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:43:09.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:43:09.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:43:12.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:43:14.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:43:16.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:43:21.940 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:43:23.092 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:43:44.490 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:43:46.499 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:45:48.897 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:45:49.695 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:45:53.965 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:45:57.051 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:45:59.456 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:45:59.456 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:45:59.456 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:45:59.456 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:45:59.456 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:45:59.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:45:59.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:46:01.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:46:02.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:46:03.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:46:04.305 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:46:06.667 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:46:29.180 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:46:29.336 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:48:30.291 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:48:34.145 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:48:36.146 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:48:39.040 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:48:41.457 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:48:41.457 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:48:41.457 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:48:41.457 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:48:41.457 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:48:41.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:48:41.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:48:42.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:48:44.116 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:48:45.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:48:47.060 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:48:48.427 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:49:09.509 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:49:10.904 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:51:11.930 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:51:15.090 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:51:16.792 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:51:18.521 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:51:19.765 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:51:19.765 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:51:19.765 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:51:21.223 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:51:21.223 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:51:21.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:51:22.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:51:24.899 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:51:26.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:51:29.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:51:30.010 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:51:33.088 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:51:54.377 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:51:56.434 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:53:57.364 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-18T02:54:00.582 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:54:08.962 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:54:10.782 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-18T02:54:12.261 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-18T02:54:12.261 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-18T02:54:12.261 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-18T02:54:12.262 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-18T02:54:12.262 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:54:12.262 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2023-03-18T02:54:12.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-03-18T02:54:13.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-18T02:54:13.932 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-03-18T02:54:15.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-18T02:54:24.549 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:54:26.623 INFO:teuthology.misc:Compressing logs...
2023-03-18T02:54:50.926 INFO:teuthology.kill:Killing Pids: {11849}
2023-03-18T02:54:51.813 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(job_info["machine_type"], job_info["status"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results_created', 'teuthology_job_results'}
2023-03-18T02:56:54.247 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-18T02:56:55.869 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-03-18T02:56:56.925 INFO:teuthology.nuke:Checking targets against current locks
2023-03-18T02:56:56.987 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-18T02:57:00.037 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-18T02:57:00.710 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-18T02:57:01.309 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-18T02:57:01.876 INFO:teuthology.orchestra.console:Power off smithi008
2023-03-18T02:57:02.932 INFO:teuthology.orchestra.console:Power off smithi098
2023-03-18T02:57:04.480 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-18T02:57:05.615 INFO:teuthology.orchestra.console:Power off smithi145
2023-03-18T02:57:06.438 INFO:teuthology.orchestra.console:Power off smithi146
2023-03-18T02:57:07.749 INFO:teuthology.orchestra.console:Power off smithi158
2023-03-18T02:57:13.895 INFO:teuthology.orchestra.console:Power off for smithi008 completed
2023-03-18T02:57:15.845 INFO:teuthology.orchestra.console:Power off for smithi098 completed
2023-03-18T02:57:16.002 INFO:teuthology.lock.ops:unlocked: smithi098.front.sepia.ceph.com
2023-03-18T02:57:19.118 INFO:teuthology.orchestra.console:Power off for smithi145 completed
2023-03-18T02:57:19.135 INFO:teuthology.lock.ops:unlocked: smithi008.front.sepia.ceph.com
2023-03-18T02:57:21.026 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com
2023-03-18T02:57:23.934 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2023-03-18T02:57:23.935 INFO:teuthology.orchestra.console:Power off for smithi158 completed
2023-03-18T02:57:24.084 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com
2023-03-18T02:57:25.835 INFO:teuthology.lock.ops:unlocked: smithi158.front.sepia.ceph.com