2023-03-20T12:48:44.951 INFO:root:teuthology version: 0.0.1.dev59+g7f7f7c3
2023-03-20T12:48:45.666 INFO:teuthology.lock.ops:Start node 'smithi132.front.sepia.ceph.com' reimaging
2023-03-20T12:48:46.162 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: reset os type and version on server
2023-03-20T12:48:46.568 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server
2023-03-20T12:48:46.591 INFO:teuthology.lock.ops:Node 'smithi132.front.sepia.ceph.com' reimaging is complete
2023-03-20T12:48:47.325 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging
2023-03-20T12:48:47.325 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server
2023-03-20T12:48:47.325 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2023-03-20T12:48:47.349 INFO:teuthology.lock.ops:Node 'smithi174.front.sepia.ceph.com' reimaging is complete
2023-03-20T12:48:47.806 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of ubuntu 20.04
2023-03-20T12:48:48.061 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of ubuntu 20.04
2023-03-20T12:48:48.690 INFO:teuthology.orchestra.console:Power off smithi132
2023-03-20T12:48:51.464 INFO:teuthology.orchestra.console:Power off smithi174
2023-03-20T12:49:01.200 INFO:teuthology.orchestra.console:Power off for smithi132 completed
2023-03-20T12:49:01.470 INFO:teuthology.orchestra.console:Power on smithi132
2023-03-20T12:49:05.343 INFO:teuthology.orchestra.console:Power off for smithi174 completed
2023-03-20T12:49:05.509 INFO:teuthology.orchestra.console:Power on smithi174
2023-03-20T12:49:06.329 INFO:teuthology.orchestra.console:Power on for smithi132 completed
2023-03-20T12:49:06.665 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish
2023-03-20T12:49:14.502 INFO:teuthology.orchestra.console:Power on for smithi174 completed
2023-03-20T12:49:15.159 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish
2023-03-20T12:53:05.414 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.174
2023-03-20T12:53:11.817 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.132
2023-03-20T12:53:16.069 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.174
2023-03-20T12:53:21.829 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.132
2023-03-20T12:53:26.821 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.174
2023-03-20T12:53:32.837 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.132
2023-03-20T12:53:37.797 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.174
2023-03-20T12:53:47.941 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.174
2023-03-20T12:54:11.558 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-20T12:54:26.678 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-20T12:54:32.673 INFO:teuthology.provision.fog.smithi132:Node is ready
2023-03-20T12:54:33.056 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com
2023-03-20T12:54:34.291 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132
2023-03-20T12:54:35.155 INFO:teuthology.provision.fog.smithi132:Deploy complete!
2023-03-20T12:54:42.746 INFO:teuthology.provision.fog.smithi174:Node is ready
2023-03-20T12:54:42.757 INFO:teuthology.orchestra.run.smithi174.stdout:smithi129
2023-03-20T12:54:43.736 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.129 smithi129 smithi129
2023-03-20T12:54:44.382 INFO:teuthology.provision.fog.smithi174:Deploy complete!
2023-03-20T12:54:45.067 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com
2023-03-20T12:54:46.379 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com
2023-03-20T12:54:46.379 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-20T12:54:46.413 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server
2023-03-20T12:54:47.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-20T12:54:48.263 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64
2023-03-20T12:54:48.815 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Ubuntu"
2023-03-20T12:54:49.827 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-03-20T12:54:49.828 INFO:teuthology.orchestra.run.smithi132.stdout:ID=ubuntu
2023-03-20T12:54:49.831 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE=debian
2023-03-20T12:54:50.670 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-03-20T12:54:51.262 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="20.04"
2023-03-20T12:54:51.262 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.ubuntu.com/"
2023-03-20T12:54:51.262 INFO:teuthology.orchestra.run.smithi132.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-03-20T12:54:51.263 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-03-20T12:54:51.263 INFO:teuthology.orchestra.run.smithi132.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-03-20T12:54:51.263 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_CODENAME=focal
2023-03-20T12:54:51.263 INFO:teuthology.orchestra.run.smithi132.stdout:UBUNTU_CODENAME=focal
2023-03-20T12:54:51.266 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server
2023-03-20T12:54:52.335 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server
2023-03-20T12:54:53.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-20T12:54:53.748 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64
2023-03-20T12:54:54.425 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu"
2023-03-20T12:54:55.773 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-03-20T12:54:55.773 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu
2023-03-20T12:54:55.773 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian
2023-03-20T12:54:55.773 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-03-20T12:54:55.773 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="20.04"
2023-03-20T12:54:55.773 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/"
2023-03-20T12:54:55.774 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-03-20T12:54:55.774 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-03-20T12:54:55.775 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-03-20T12:54:55.776 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=focal
2023-03-20T12:54:55.776 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=focal
2023-03-20T12:54:55.776 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2023-03-20T12:54:55.881 INFO:teuthology.dispatcher.supervisor:Running job 7212741
2023-03-20T12:54:56.630 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-03-17_23:47:02-rbd-reef-distro-default-smithi/7212741 --name yuriw-2023-03-17_23:47:02-rbd-reef-distro-default-smithi --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} -- /home/teuthworker/archive/yuriw-2023-03-17_23:47:02-rbd-reef-distro-default-smithi/7212741/orig.config.yaml
2023-03-20T12:54:56.634 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-03-17_23:47:02-rbd-reef-distro-default-smithi/7212741
2023-03-20T12:54:57.610 INFO:teuthology.dispatcher.supervisor:Job PID: 15716
2023-03-20T12:54:57.610 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-21T00:55:50.991 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T00:55:53.204 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T00:55:56.452 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T00:55:58.546 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T00:55:58.546 INFO:teuthology.misc:Compressing logs...
2023-03-21T00:55:58.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T00:55:58.719 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T00:56:01.183 INFO:teuthology.misc:Compressing logs...
2023-03-21T00:56:02.210 INFO:teuthology.misc:Compressing logs...
2023-03-21T00:56:06.777 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/cluster2-mgr.x.log: file size changed while zipping
2023-03-21T00:56:12.763 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/cluster2-client.mirror.2.42545.log: file size changed while zipping
2023-03-21T00:56:15.543 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/cluster2-client.mirror.3.34298.log: file size changed while zipping
2023-03-21T00:57:35.173 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/cluster2-osd.0.log: file size changed while zipping
2023-03-21T00:58:43.503 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/cluster2-osd.1.log: file size changed while zipping
2023-03-21T00:59:00.211 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/cluster2-mon.a.log: file size changed while zipping
2023-03-21T01:00:06.652 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/cluster2-osd.2.log: file size changed while zipping
2023-03-21T01:01:22.573 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:01:28.181 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:03:30.688 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:03:32.773 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:03:33.727 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:03:35.792 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:03:37.915 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:03:37.915 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:03:37.915 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:03:38.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:03:38.944 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:03:39.637 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:04:54.132 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:04:55.659 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:06:58.077 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:06:59.620 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:07:01.463 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:07:03.769 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:07:06.929 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:07:06.929 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:07:06.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:07:08.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:07:10.644 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:07:12.574 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:08:26.096 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:08:28.141 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:10:29.424 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:10:31.470 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:10:33.125 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:10:36.136 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:10:37.758 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:10:37.759 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:10:38.789 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:10:40.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:10:42.545 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:10:44.509 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:11:55.766 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:11:56.805 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:13:59.422 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:14:01.954 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:14:05.912 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:14:07.092 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:14:08.159 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:14:08.159 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:14:08.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:14:08.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:14:10.085 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:14:11.611 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:15:25.199 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:15:27.765 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:17:31.309 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:17:32.541 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:17:34.516 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:17:37.348 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:17:39.233 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:17:39.233 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:17:39.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:17:39.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:17:40.605 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:17:42.628 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:18:54.825 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:18:58.197 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:21:00.694 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:21:02.382 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:21:03.911 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:21:06.128 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:21:07.557 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:21:07.557 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:21:07.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:21:07.732 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:21:08.823 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:21:11.467 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:22:27.608 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:22:30.213 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:24:33.351 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:24:34.739 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:24:36.066 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:24:37.420 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:24:38.804 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:24:39.979 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:24:39.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:24:40.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:24:42.101 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:24:43.331 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:25:55.149 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:25:56.265 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:27:57.693 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:27:58.741 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:28:00.332 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:28:01.752 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:28:03.374 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:28:03.374 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:28:03.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:28:03.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:28:04.463 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:28:06.030 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:29:19.102 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:29:20.387 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:31:22.068 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:31:23.943 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:31:25.848 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:31:26.828 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:31:29.079 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:31:29.079 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:31:29.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:31:29.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:31:31.396 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:31:32.689 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:32:43.681 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:32:45.215 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:34:46.976 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:34:51.393 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:34:54.728 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:34:57.101 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:34:57.752 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:34:57.752 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:34:58.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:34:58.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:34:59.431 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:35:00.252 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:36:14.276 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:36:15.957 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:38:18.111 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:38:20.316 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:38:22.129 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:38:22.952 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:38:26.367 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:38:26.367 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:38:26.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:38:26.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:38:28.953 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:38:31.355 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:39:42.559 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:39:45.522 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:41:46.216 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:41:48.195 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:41:49.000 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:41:50.084 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:41:51.285 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:41:51.285 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:41:51.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:41:51.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:41:53.477 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:41:55.685 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:43:07.720 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:43:09.390 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:45:10.829 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:45:11.730 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:45:23.209 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:45:24.487 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:45:26.736 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:45:26.736 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:45:26.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:45:26.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:45:28.362 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:45:31.491 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:46:51.551 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:46:52.778 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:48:54.291 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:48:57.668 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:48:58.584 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:49:00.023 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:49:01.981 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:49:01.981 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:49:01.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:49:02.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:49:03.437 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:49:05.765 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:50:23.344 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:50:25.030 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:52:26.602 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:52:28.527 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:52:30.271 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:52:32.034 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:52:33.019 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:52:33.019 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:52:33.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:52:33.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:52:34.877 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:52:38.006 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:53:50.942 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:53:52.326 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:55:54.052 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:55:55.074 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:55:58.212 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:55:59.983 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:56:01.780 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:56:01.780 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:56:01.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:56:01.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:56:04.356 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:56:05.155 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:57:16.729 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:57:17.638 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:59:19.898 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T01:59:22.304 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T01:59:23.482 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T01:59:25.439 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T01:59:26.302 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T01:59:26.302 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:59:27.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T01:59:28.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T01:59:29.865 INFO:teuthology.misc:Compressing logs...
2023-03-21T01:59:32.111 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:00:44.357 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:00:45.246 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:02:47.185 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:02:48.137 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:02:49.348 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:02:51.790 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:02:53.167 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:02:53.167 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:02:53.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:02:53.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:02:55.098 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:02:58.569 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:04:10.858 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:04:15.143 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:06:16.916 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:06:19.319 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:06:20.916 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:06:22.264 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:06:23.078 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:06:23.078 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:06:23.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:06:23.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:06:25.007 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:06:26.216 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:07:41.767 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:07:43.445 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:09:46.953 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:09:56.065 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:09:57.358 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:09:59.853 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:10:03.137 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:10:03.137 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:10:03.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:10:03.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:10:04.811 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:10:07.713 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:11:20.227 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:11:21.719 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:13:23.161 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:13:25.885 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:13:28.720 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:13:30.348 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:13:31.620 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:13:31.620 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:13:31.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:13:31.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:13:33.510 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:13:35.342 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:14:47.199 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:14:50.185 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:16:51.825 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:16:52.503 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:16:54.502 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:16:56.725 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:16:58.226 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:16:58.226 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:16:58.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:16:58.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:17:01.159 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:17:02.854 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:18:14.238 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:18:15.743 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:20:18.258 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:20:19.993 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:20:21.868 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:20:23.007 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:20:35.985 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:20:35.985 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:20:35.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:20:36.156 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:20:38.157 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:20:40.288 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:21:57.019 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:22:05.069 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:24:07.017 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:24:08.782 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:24:10.063 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:24:11.391 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:24:12.542 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:24:12.542 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:24:12.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:24:13.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:24:14.993 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:24:17.924 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:25:34.961 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:25:35.702 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:27:38.445 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:27:40.514 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:27:43.537 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:27:45.937 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:27:48.651 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:27:48.652 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:27:48.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:27:48.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:27:50.649 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:27:52.559 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:29:04.546 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:29:06.062 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:31:08.257 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:31:09.808 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:31:13.378 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:31:15.066 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:31:17.023 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:31:17.023 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:31:17.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:31:17.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:31:24.141 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:31:26.208 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:32:39.280 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:32:40.502 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:34:52.689 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:34:53.879 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:34:56.088 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:35:07.242 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:35:08.403 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:35:08.403 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:35:08.404 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:35:08.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:35:10.945 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:35:24.862 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:36:42.864 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:36:43.389 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:38:44.320 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:38:48.205 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:38:52.502 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:38:54.972 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:38:56.369 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:38:56.369 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:38:56.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:38:56.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:39:02.127 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:39:03.081 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:40:16.059 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:40:19.234 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:42:19.873 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:42:20.910 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:42:22.271 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:42:23.837 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:42:25.437 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:42:25.437 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:42:25.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:42:25.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:42:28.724 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:42:30.809 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:43:44.927 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:43:48.569 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:45:52.365 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:45:53.657 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:45:55.116 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:45:56.355 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:45:58.347 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:45:58.347 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:45:58.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:45:58.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:46:02.698 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:46:04.658 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:47:17.345 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:47:18.484 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:49:20.281 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-21T02:49:22.021 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:49:23.573 ERROR:teuthology.dispatcher.supervisor:Failed to kill job
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 288, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:49:25.666 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2023-03-21T02:49:28.097 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2023-03-21T02:49:28.097 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:49:28.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2023-03-21T02:49:28.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2023-03-21T02:49:30.657 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:49:32.387 INFO:teuthology.misc:Compressing logs...
2023-03-21T02:50:43.721 INFO:teuthology.kill:Killing Pids: {15716}
2023-03-21T02:50:45.960 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/dispatcher/supervisor.py", line 302, in run_with_watchdog
    kill_job(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/kill.py", line 90, in kill_job
    teuthology.exporter.JobResults().record(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/teuthology/exporter.py", line 175, in __init__
    self.metric = Counter(
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/metrics.py", line 143, in __init__
    registry.register(self)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7f7f7c33a372789d0252bbbcd5798300b0f044d0/virtualenv/lib/python3.8/site-packages/prometheus_client/registry.py", line 43, in register
    raise ValueError(
ValueError: Duplicated timeseries in CollectorRegistry: {'teuthology_job_results_created', 'teuthology_job_results_total', 'teuthology_job_results'}
2023-03-21T02:52:48.924 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-21T02:52:50.846 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-03-21T02:52:53.536 INFO:teuthology.nuke:Checking targets against current locks
2023-03-21T02:52:53.594 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-21T02:52:56.819 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-21T02:52:59.944 INFO:teuthology.orchestra.console:Power off smithi132
2023-03-21T02:53:01.311 INFO:teuthology.orchestra.console:Power off smithi174
2023-03-21T02:53:10.636 INFO:teuthology.orchestra.console:Power off for smithi132 completed
2023-03-21T02:53:12.229 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com
2023-03-21T02:53:14.756 INFO:teuthology.orchestra.console:Power off for smithi174 completed
2023-03-21T02:53:16.053 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com