2023-03-18T21:06:29.407 INFO:root:teuthology version: 0.0.1.dev59+g7f7f7c3
2023-03-18T21:06:32.836 INFO:teuthology.lock.ops:Start node 'smithi052.front.sepia.ceph.com' reimaging
2023-03-18T21:06:35.504 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: reset os type and version on server
2023-03-18T21:06:36.347 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2023-03-18T21:06:36.370 INFO:teuthology.lock.ops:Node 'smithi052.front.sepia.ceph.com' reimaging is complete
2023-03-18T21:06:37.211 INFO:teuthology.lock.ops:Start node 'smithi101.front.sepia.ceph.com' reimaging
2023-03-18T21:06:37.211 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: reset os type and version on server
2023-03-18T21:06:39.310 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2023-03-18T21:06:41.403 INFO:teuthology.lock.ops:Node 'smithi101.front.sepia.ceph.com' reimaging is complete
2023-03-18T21:06:42.633 INFO:teuthology.lock.ops:Start node 'smithi102.front.sepia.ceph.com' reimaging
2023-03-18T21:06:42.633 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: reset os type and version on server
2023-03-18T21:06:42.633 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server
2023-03-18T21:06:42.661 INFO:teuthology.lock.ops:Node 'smithi102.front.sepia.ceph.com' reimaging is complete
2023-03-18T21:06:44.785 INFO:teuthology.lock.ops:Start node 'smithi141.front.sepia.ceph.com' reimaging
2023-03-18T21:06:44.786 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: reset os type and version on server
2023-03-18T21:06:44.786 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server
2023-03-18T21:06:44.819 INFO:teuthology.lock.ops:Node 'smithi141.front.sepia.ceph.com' reimaging is complete
2023-03-18T21:06:46.969 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging
2023-03-18T21:06:46.969 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server
2023-03-18T21:06:46.969 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2023-03-18T21:06:47.002 INFO:teuthology.lock.ops:Node 'smithi181.front.sepia.ceph.com' reimaging is complete
2023-03-18T21:06:48.411 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of centos 8.stream
2023-03-18T21:06:50.071 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of centos 8.stream
2023-03-18T21:06:55.372 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of centos 8.stream
2023-03-18T21:06:57.065 INFO:teuthology.provision.fog.smithi141:Scheduling deploy of centos 8.stream
2023-03-18T21:06:59.946 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 8.stream
2023-03-18T21:07:01.963 INFO:teuthology.orchestra.console:Power off smithi052
2023-03-18T21:07:03.881 INFO:teuthology.orchestra.console:Power off smithi102
2023-03-18T21:07:05.083 INFO:teuthology.orchestra.console:Power off smithi141
2023-03-18T21:07:07.044 INFO:teuthology.orchestra.console:Power off smithi101
2023-03-18T21:07:08.024 INFO:teuthology.orchestra.console:Power off smithi181
2023-03-18T21:07:08.759 INFO:teuthology.orchestra.console:Power off for smithi102 completed
2023-03-18T21:07:10.269 INFO:teuthology.orchestra.console:Power off for smithi141 completed
2023-03-18T21:07:10.370 INFO:teuthology.orchestra.console:Power on smithi102
2023-03-18T21:07:11.768 INFO:teuthology.orchestra.console:Power on smithi141
2023-03-18T21:07:13.851 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2023-03-18T21:07:15.868 INFO:teuthology.orchestra.console:Power on smithi181
2023-03-18T21:07:17.825 INFO:teuthology.orchestra.console:Power off for smithi052 completed
2023-03-18T21:07:20.787 INFO:teuthology.orchestra.console:Power on smithi052
2023-03-18T21:07:24.978 INFO:teuthology.orchestra.console:Power on for smithi141 completed
2023-03-18T21:07:26.909 INFO:teuthology.orchestra.console:Power on for smithi102 completed
2023-03-18T21:07:26.909 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2023-03-18T21:07:27.010 INFO:teuthology.provision.fog.smithi141:Waiting for deploy to finish
2023-03-18T21:07:29.945 INFO:teuthology.orchestra.console:Power on for smithi181 completed
2023-03-18T21:07:29.946 INFO:teuthology.provision.fog.smithi102:Waiting for deploy to finish
2023-03-18T21:07:29.946 INFO:teuthology.orchestra.console:Power on smithi101
2023-03-18T21:07:30.046 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish
2023-03-18T21:07:31.365 INFO:teuthology.orchestra.console:Power on for smithi052 completed
2023-03-18T21:07:32.405 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish
2023-03-18T21:07:38.314 INFO:teuthology.orchestra.console:Power on for smithi101 completed
2023-03-18T21:07:39.953 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish
2023-03-18T21:10:02.096 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: Authentication failed.
2023-03-18T21:10:03.781 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-18T21:10:08.741 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2023-03-18T21:10:14.693 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-18T21:10:18.821 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.141
2023-03-18T21:10:20.125 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2023-03-18T21:10:24.709 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-18T21:10:29.381 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.141
2023-03-18T21:10:31.637 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2023-03-18T21:10:31.638 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-18T21:10:35.365 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-18T21:10:40.709 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2023-03-18T21:10:41.271 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-18T21:10:43.118 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.141
2023-03-18T21:10:46.661 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-18T21:10:51.525 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2023-03-18T21:10:52.197 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-18T21:10:52.653 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.141
2023-03-18T21:11:01.733 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.141
2023-03-18T21:11:03.595 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-18T21:11:05.615 WARNING:teuthology.provision.fog:timed out
2023-03-18T21:11:12.677 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-18T21:11:28.005 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.141
2023-03-18T21:11:37.989 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-03-18T21:11:39.201 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.141
2023-03-18T21:11:45.511 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-18T21:11:47.214 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-18T21:11:54.469 WARNING:teuthology.provision.fog:timed out
2023-03-18T21:11:57.556 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-18T21:11:58.760 WARNING:teuthology.provision.fog:timed out
2023-03-18T21:12:02.571 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-18T21:12:06.235 INFO:teuthology.provision.fog.smithi141:Node is ready
2023-03-18T21:12:06.259 INFO:teuthology.orchestra.run.smithi141.stdout:smithi141.front.sepia.ceph.com
2023-03-18T21:12:08.468 INFO:teuthology.provision.fog.smithi181:Node is ready
2023-03-18T21:12:08.468 INFO:teuthology.provision.fog.smithi102:Node is ready
2023-03-18T21:12:08.486 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102.front.sepia.ceph.com
2023-03-18T21:12:11.021 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com
2023-03-18T21:12:11.021 INFO:teuthology.orchestra.run.smithi141.stdout:172.21.15.141 smithi141.front.sepia.ceph.com smithi141
2023-03-18T21:12:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102 smithi102.front.sepia.ceph.com smithi102
2023-03-18T21:12:13.230 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-18T21:12:13.248 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181
2023-03-18T21:12:15.966 INFO:teuthology.provision.fog.smithi101:Node is ready
2023-03-18T21:12:15.967 INFO:teuthology.provision.fog.smithi052:Node is ready
2023-03-18T21:12:17.752 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com
2023-03-18T21:12:19.677 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052.front.sepia.ceph.com
2023-03-18T21:12:22.863 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101
2023-03-18T21:12:25.270 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52 smithi052.front.sepia.ceph.com smithi052
2023-03-18T21:12:25.403 INFO:teuthology.provision.fog.smithi102:Deploy complete!
2023-03-18T21:12:27.133 INFO:teuthology.provision.fog.smithi141:Deploy complete!
2023-03-18T21:12:29.810 INFO:teuthology.provision.fog.smithi181:Deploy complete!
2023-03-18T21:12:31.334 INFO:teuthology.provision.fog.smithi101:Deploy complete!
2023-03-18T21:12:33.571 INFO:teuthology.provision.fog.smithi052:Deploy complete!
2023-03-18T21:12:38.612 INFO:teuthology.lock.ops:Checking smithi052.front.sepia.ceph.com
2023-03-18T21:12:40.057 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-18T21:12:40.081 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com
2023-03-18T21:12:41.278 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-18T21:12:41.305 INFO:teuthology.lock.ops:Checking smithi102.front.sepia.ceph.com
2023-03-18T21:12:42.297 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-18T21:12:42.321 INFO:teuthology.lock.ops:Checking smithi141.front.sepia.ceph.com
2023-03-18T21:12:43.402 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-18T21:12:43.430 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com
2023-03-18T21:12:45.610 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-18T21:12:45.639 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: set os type and version on server
2023-03-18T21:12:46.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-18T21:12:46.710 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64
2023-03-18T21:12:47.224 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream"
2023-03-18T21:12:48.044 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="8"
2023-03-18T21:12:48.044 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos"
2023-03-18T21:12:48.044 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora"
2023-03-18T21:12:48.044 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="8"
2023-03-18T21:12:48.044 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el8"
2023-03-18T21:12:48.044 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-18T21:12:48.044 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31"
2023-03-18T21:12:48.044 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-18T21:12:48.045 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/"
2023-03-18T21:12:48.045 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-18T21:12:48.045 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-18T21:12:48.045 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-18T21:12:48.046 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2023-03-18T21:12:48.763 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server
2023-03-18T21:12:49.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-18T21:12:49.962 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64
2023-03-18T21:12:50.742 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="CentOS Stream"
2023-03-18T21:12:52.345 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8"
2023-03-18T21:12:52.345 INFO:teuthology.orchestra.run.smithi101.stdout:ID="centos"
2023-03-18T21:12:52.345 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="rhel fedora"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://centos.org/"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-18T21:12:52.346 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-18T21:12:52.347 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2023-03-18T21:12:52.378 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: set os type and version on server
2023-03-18T21:12:53.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-18T21:12:53.662 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64
2023-03-18T21:12:54.480 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream"
2023-03-18T21:12:56.246 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8"
2023-03-18T21:12:56.246 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos"
2023-03-18T21:12:56.246 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora"
2023-03-18T21:12:56.246 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8"
2023-03-18T21:12:56.247 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8"
2023-03-18T21:12:56.247 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-18T21:12:56.247 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31"
2023-03-18T21:12:56.247 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-18T21:12:56.247 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/"
2023-03-18T21:12:56.247 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-18T21:12:56.247 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-18T21:12:56.247 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-18T21:12:56.247 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server
2023-03-18T21:12:56.845 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: set os type and version on server
2023-03-18T21:12:58.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-18T21:12:58.934 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64
2023-03-18T21:13:00.982 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="CentOS Stream"
2023-03-18T21:13:01.995 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8"
2023-03-18T21:13:01.996 INFO:teuthology.orchestra.run.smithi141.stdout:ID="centos"
2023-03-18T21:13:01.996 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="rhel fedora"
2023-03-18T21:13:01.996 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8"
2023-03-18T21:13:01.996 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8"
2023-03-18T21:13:02.947 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-18T21:13:02.947 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31"
2023-03-18T21:13:02.947 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-18T21:13:02.947 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://centos.org/"
2023-03-18T21:13:02.947 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-18T21:13:02.948 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-18T21:13:02.948 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-18T21:13:02.948 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server
2023-03-18T21:13:02.970 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server
2023-03-18T21:13:03.517 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-18T21:13:03.767 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64
2023-03-18T21:13:05.725 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream"
2023-03-18T21:13:08.225 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8"
2023-03-18T21:13:08.225 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos"
2023-03-18T21:13:08.225 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora"
2023-03-18T21:13:08.225 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8"
2023-03-18T21:13:08.225 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8"
2023-03-18T21:13:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-18T21:13:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31"
2023-03-18T21:13:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-18T21:13:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/"
2023-03-18T21:13:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-18T21:13:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-18T21:13:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-18T21:13:10.370 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2023-03-18T21:13:13.133 INFO:teuthology.dispatcher.supervisor:Running job 7211168
2023-03-18T21:13:13.879 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-03-16_21:59:27-rados-wip-yuri6-testing-2023-03-12-0918-pacific-distro-default-smithi/7211168 --name yuriw-2023-03-16_21:59:27-rados-wip-yuri6-testing-2023-03-12-0918-pacific-distro-default-smithi --description rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} -- /home/teuthworker/archive/yuriw-2023-03-16_21:59:27-rados-wip-yuri6-testing-2023-03-12-0918-pacific-distro-default-smithi/7211168/orig.config.yaml
2023-03-18T21:13:13.882 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-03-16_21:59:27-rados-wip-yuri6-testing-2023-03-12-0918-pacific-distro-default-smithi/7211168
2023-03-18T21:13:14.668 INFO:teuthology.dispatcher.supervisor:Job PID: 21898
2023-03-18T21:13:14.668 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-19T09:13:38.712 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:13:45.437 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:13:48.644 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:13:50.525 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:13:50.525 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:13:50.525 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:13:50.525 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:13:50.525 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:13:50.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:13:56.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:13:58.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:13:59.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:14:02.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:14:04.870 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:14:07.570 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-19T09:14:09.364 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-19T09:14:10.183 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-19T09:14:10.183 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-19T09:14:10.839 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:14:18.791 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-mon.smithi141.log: file size changed while zipping
2023-03-19T09:14:25.252 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2023-03-19T09:14:25.253 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2023-03-19T09:14:25.253 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-mon.smithi181.log: file size changed while zipping
2023-03-19T09:14:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.14.log: file size changed while zipping
2023-03-19T09:14:28.326 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.8.log: file size changed while zipping
2023-03-19T09:14:29.405 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.9.log: file size changed while zipping
2023-03-19T09:14:31.346 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2023-03-19T09:14:34.071 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.12.log: file size changed while zipping
2023-03-19T09:14:36.049 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.11.log: file size changed while zipping
2023-03-19T09:14:38.687 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2023-03-19T09:14:44.067 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.13.log: file size changed while zipping
2023-03-19T09:14:44.067 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/a4a0770e-c5d5-11ed-9afc-001a4aab830c/ceph-osd.10.log: file size changed while zipping
2023-03-19T09:15:08.822 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:15:09.459 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:17:10.929 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:17:12.811 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:17:14.496 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:17:18.199 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:17:20.705 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:17:22.250 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:17:22.250 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:17:22.250 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:17:22.250 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:17:22.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:17:22.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:17:24.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:17:26.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:17:27.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:17:29.917 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:17:34.085 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:17:59.345 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:18:00.054 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:20:01.384 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:20:02.392 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:20:03.654 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:20:05.643 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:20:06.841 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:20:06.841 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:20:06.841 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:20:06.841 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:20:06.841 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:20:06.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:20:07.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:20:10.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:20:11.995 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:20:13.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:20:15.531 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:20:17.514 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:20:42.614 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:20:43.655 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:22:44.759 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:22:46.112 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:22:46.965 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:22:50.170 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:22:50.998 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:22:50.998 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:22:50.998 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:22:50.998 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:22:50.998 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:22:50.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:22:51.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:22:53.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:22:54.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:22:58.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:23:01.654 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:23:06.903 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:23:34.144 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:23:39.965 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:25:41.605 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:25:43.067 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:25:47.594 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:25:48.988 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:25:51.056 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:25:51.056 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:25:51.056 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:25:51.056 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:25:51.056 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:25:51.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:25:51.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:25:55.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:25:57.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:26:00.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:26:10.363 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:26:14.391 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:26:41.070 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:26:45.916 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:28:48.400 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:28:56.661 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:28:58.632 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:29:00.596 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:29:12.068 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:29:12.068 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:29:12.068 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:29:12.069 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:29:12.069 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:29:12.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:29:12.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:29:14.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:29:17.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:29:19.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:29:21.749 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:29:23.781 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:29:49.693 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:29:50.684 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:31:55.613 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:31:58.083 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:32:01.003 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:32:03.119 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:32:05.132 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:32:05.133 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:32:05.133 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:32:05.133 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:32:05.133 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:32:05.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:32:05.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:32:07.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:32:08.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:32:17.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:32:25.242 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:32:27.055 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:32:52.520 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:32:54.029 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:34:55.490 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:34:56.879 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:34:58.008 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:34:59.316 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:35:02.647 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:35:02.647 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:35:02.647 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:35:02.647 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:35:02.647 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:35:02.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:35:04.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:35:06.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:35:07.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:35:09.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:35:10.246 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:35:11.976 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:35:37.185 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:35:39.595 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:37:40.784 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:37:43.527 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:37:45.754 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:37:48.057 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:37:50.435 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:37:52.960 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:37:52.960 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:37:52.960 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:37:53.470 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:37:54.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:37:55.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:37:56.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:37:57.012 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:38:00.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:38:04.357 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:38:06.457 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:38:31.822 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:38:32.513 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:40:33.705 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:40:35.451 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:40:36.982 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:40:41.462 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:40:42.486 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:40:42.486 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:40:42.486 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:40:42.487 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:40:42.487 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:40:43.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:40:43.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:40:45.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:40:47.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:40:49.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:40:51.803 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:40:55.018 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:41:20.921 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:41:21.876 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:43:23.147 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:43:24.218 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:43:25.830 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:43:27.824 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:43:28.904 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:43:28.904 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:43:28.904 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:43:28.904 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:43:28.905 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:43:28.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:43:29.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:43:29.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:43:30.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:43:31.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:43:34.733 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:43:36.742 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:44:04.725 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:44:06.207 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:46:07.581 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:46:10.129 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:46:11.244 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:46:12.732 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:46:13.848 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:46:13.848 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:46:13.848 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:46:13.848 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:46:13.848 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:46:13.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:46:14.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:46:16.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:46:17.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:46:21.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:46:26.171 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:46:31.508 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:46:56.806 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:46:58.104 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:49:01.270 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:49:03.436 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:49:04.254 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:49:05.781 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:49:07.497 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:49:07.497 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:49:07.497 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:49:09.784 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:49:09.784 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:49:09.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:49:09.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:49:12.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:49:13.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:49:15.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:49:17.831 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:49:20.779 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:49:46.947 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:49:49.181 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:51:52.451 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:51:54.549 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:51:55.647 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:51:57.960 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:51:59.347 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:51:59.347 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:51:59.347 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:51:59.348 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:51:59.348 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:51:59.348 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:51:59.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:52:00.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:52:01.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:52:03.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:52:05.193 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:52:10.038 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:52:36.031 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:52:39.013 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:54:41.535 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:54:42.519 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:54:43.487 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:54:44.990 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:54:46.100 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:54:46.100 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:54:46.100 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:54:46.101 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:54:46.101 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:54:47.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:54:47.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:54:49.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:54:50.060 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:54:51.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:54:55.294 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:54:57.068 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:55:24.929 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:55:28.901 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:57:30.742 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T09:57:32.648 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:57:33.761 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T09:57:35.267 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T09:57:36.897 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T09:57:36.897 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T09:57:36.897 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T09:57:36.897 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T09:57:36.897 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:57:36.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T09:57:37.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T09:57:38.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T09:57:41.363 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T09:57:42.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T09:57:43.672 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:57:45.378 INFO:teuthology.misc:Compressing logs...
2023-03-19T09:58:09.951 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T09:58:10.261 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:00:10.725 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:00:13.487 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:00:15.136 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:00:16.663 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:00:19.549 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:00:19.549 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:00:19.549 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:00:19.549 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:00:19.549 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:00:19.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:00:19.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:00:21.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:00:22.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:00:23.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:00:26.403 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:00:29.456 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:00:56.525 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:00:57.791 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:03:00.145 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:03:00.910 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:03:02.365 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:03:04.637 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:03:05.945 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:03:05.945 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:03:05.945 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:03:05.945 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:03:05.945 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:03:05.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:03:06.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:03:07.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:03:09.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:03:14.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:03:15.266 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:03:17.877 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:03:44.016 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:03:45.644 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:05:47.626 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:05:48.206 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:05:49.485 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:05:50.282 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:05:51.909 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:05:51.910 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:05:51.910 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:05:51.910 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:05:51.910 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:05:51.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:05:52.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:05:53.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:05:54.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:05:56.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:05:58.520 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:06:01.672 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:06:27.249 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:06:30.016 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:08:33.209 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:08:33.824 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:08:34.657 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:08:35.181 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:08:36.144 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:08:36.144 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:08:36.144 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:08:36.144 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:08:36.144 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:08:36.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:08:36.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:08:38.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:08:40.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:08:42.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:08:43.995 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:08:45.960 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:09:12.062 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:09:13.818 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:11:16.624 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:11:18.196 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:11:19.367 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:11:20.536 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:11:21.870 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:11:21.871 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:11:21.871 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:11:21.871 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:11:21.871 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:11:21.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:11:22.055 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:11:24.708 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:11:27.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:11:28.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:11:33.704 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:11:37.124 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:12:05.172 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:12:07.364 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:14:09.697 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:14:11.960 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:14:14.741 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:14:16.504 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:14:19.649 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:14:19.650 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:14:19.650 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:14:19.650 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:14:19.650 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:14:19.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:14:19.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:14:21.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:14:22.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:14:23.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:14:24.339 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:14:25.668 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:14:52.233 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:14:54.508 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:16:55.424 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:16:56.258 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:16:57.586 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:16:58.448 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:17:00.256 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:17:03.161 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:17:04.721 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:17:04.721 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:17:04.721 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:17:04.721 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:17:04.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:17:06.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:17:10.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:17:13.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:17:14.975 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:17:16.653 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:17:43.047 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:17:43.528 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:19:45.286 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:19:46.283 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:19:47.887 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:19:48.686 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:19:49.316 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:19:51.189 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:19:51.190 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:19:51.190 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:19:51.190 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:19:51.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:19:51.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:19:52.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:19:53.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:19:57.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:19:59.600 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:20:01.061 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:20:26.921 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:20:27.234 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:22:28.509 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-19T10:22:29.701 INFO:teuthology.kill:Killing Pids: {21898}
2023-03-19T10:22:31.825 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:22:33.768 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
2023-03-19T10:22:37.235 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
2023-03-19T10:22:37.235 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
2023-03-19T10:22:37.236 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
2023-03-19T10:22:37.236 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
2023-03-19T10:22:37.236 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:22:37.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2023-03-19T10:22:37.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-19T10:22:39.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-19T10:22:41.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2023-03-19T10:22:42.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2023-03-19T10:22:43.789 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:22:45.186 INFO:teuthology.misc:Compressing logs...
2023-03-19T10:23:11.300 INFO:teuthology.kill:No teuthology processes running
2023-03-19T10:23:12.908 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_total', 'teuthology_job_results', 'teuthology_job_results_created'}
2023-03-19T10:25:13.041 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-19T10:25:19.719 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-03-19T10:25:20.952 INFO:teuthology.nuke:Checking targets against current locks
2023-03-19T10:25:22.343 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-19T10:25:25.081 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-19T10:25:26.933 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-19T10:25:27.959 INFO:teuthology.orchestra.console:Power off smithi052
2023-03-19T10:25:29.392 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-19T10:25:30.667 INFO:teuthology.orchestra.console:Power off smithi101
2023-03-19T10:25:32.260 INFO:teuthology.orchestra.console:Power off smithi102
2023-03-19T10:25:33.415 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-19T10:25:35.444 INFO:teuthology.orchestra.console:Power off smithi141
2023-03-19T10:25:36.863 INFO:teuthology.orchestra.console:Power off smithi181
2023-03-19T10:25:39.571 INFO:teuthology.orchestra.console:Power off for smithi052 completed
2023-03-19T10:25:40.694 INFO:teuthology.lock.ops:unlocked: smithi052.front.sepia.ceph.com
2023-03-19T10:25:43.703 INFO:teuthology.orchestra.console:Power off for smithi102 completed
2023-03-19T10:25:44.387 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2023-03-19T10:25:46.978 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com
2023-03-19T10:25:51.911 INFO:teuthology.lock.ops:unlocked: smithi101.front.sepia.ceph.com
2023-03-19T10:25:54.193 INFO:teuthology.orchestra.console:Power off for smithi141 completed
2023-03-19T10:25:55.526 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2023-03-19T10:25:55.684 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com
2023-03-19T10:25:56.398 INFO:teuthology.lock.ops:unlocked: smithi141.front.sepia.ceph.com