2022-10-16T17:42:18.626 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-16T17:42:18.765 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging 2022-10-16T17:42:18.766 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server 2022-10-16T17:42:18.767 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2022-10-16T17:42:18.785 INFO:teuthology.lock.ops:Node 'smithi027.front.sepia.ceph.com' reimaging is complete 2022-10-16T17:42:18.785 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging 2022-10-16T17:42:18.786 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server 2022-10-16T17:42:18.786 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2022-10-16T17:42:18.805 INFO:teuthology.lock.ops:Node 'smithi050.front.sepia.ceph.com' reimaging is complete 2022-10-16T17:42:19.001 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of ubuntu 20.04 2022-10-16T17:42:19.035 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of ubuntu 20.04 2022-10-16T17:42:19.535 INFO:teuthology.orchestra.console:Power off smithi050 2022-10-16T17:42:19.552 INFO:teuthology.orchestra.console:Power off smithi027 2022-10-16T17:42:27.912 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2022-10-16T17:42:28.013 INFO:teuthology.orchestra.console:Power on smithi050 2022-10-16T17:42:32.031 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2022-10-16T17:42:32.132 INFO:teuthology.orchestra.console:Power on smithi027 2022-10-16T17:42:36.380 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2022-10-16T17:42:36.481 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2022-10-16T17:42:40.496 INFO:teuthology.orchestra.console:Power on for smithi027 completed 2022-10-16T17:42:40.598 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish 2022-10-16T17:47:34.051 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-16T17:47:34.583 INFO:teuthology.provision.fog.smithi050:Node is ready 2022-10-16T17:47:34.590 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2022-10-16T17:47:34.598 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2022-10-16T17:47:34.757 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2022-10-16T17:49:56.672 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-16T17:49:57.185 INFO:teuthology.provision.fog.smithi027:Node is ready 2022-10-16T17:49:57.231 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com 2022-10-16T17:49:57.239 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027 2022-10-16T17:49:57.457 INFO:teuthology.provision.fog.smithi027:Deploy complete! 2022-10-16T17:49:57.780 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com 2022-10-16T17:49:57.781 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-16T17:49:57.807 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2022-10-16T17:49:57.809 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-16T17:49:57.836 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server 2022-10-16T17:49:57.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2022-10-16T17:49:58.135 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2022-10-16T17:49:58.157 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="Ubuntu" 2022-10-16T17:49:58.158 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-10-16T17:49:58.158 INFO:teuthology.orchestra.run.smithi027.stdout:ID=ubuntu 2022-10-16T17:49:58.159 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE=debian 2022-10-16T17:49:58.159 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-10-16T17:49:58.160 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="20.04" 2022-10-16T17:49:58.160 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://www.ubuntu.com/" 2022-10-16T17:49:58.161 INFO:teuthology.orchestra.run.smithi027.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-10-16T17:49:58.161 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-10-16T17:49:58.162 INFO:teuthology.orchestra.run.smithi027.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-10-16T17:49:58.162 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_CODENAME=focal 2022-10-16T17:49:58.163 INFO:teuthology.orchestra.run.smithi027.stdout:UBUNTU_CODENAME=focal 2022-10-16T17:49:58.165 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2022-10-16T17:49:58.193 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2022-10-16T17:49:58.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2022-10-16T17:49:58.485 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2022-10-16T17:49:58.502 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Ubuntu" 2022-10-16T17:49:58.503 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-10-16T17:49:58.503 INFO:teuthology.orchestra.run.smithi050.stdout:ID=ubuntu 2022-10-16T17:49:58.504 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE=debian 2022-10-16T17:49:58.504 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-10-16T17:49:58.505 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="20.04" 2022-10-16T17:49:58.505 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.ubuntu.com/" 2022-10-16T17:49:58.505 INFO:teuthology.orchestra.run.smithi050.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-10-16T17:49:58.506 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-10-16T17:49:58.506 INFO:teuthology.orchestra.run.smithi050.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-10-16T17:49:58.507 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_CODENAME=focal 2022-10-16T17:49:58.507 INFO:teuthology.orchestra.run.smithi050.stdout:UBUNTU_CODENAME=focal 2022-10-16T17:49:58.508 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2022-10-16T17:49:58.636 INFO:teuthology.dispatcher.supervisor:Running job 7069133 2022-10-16T17:49:58.637 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_amathuri@teuthology --archive /home/teuthworker/archive/amathuri-2022-10-16_17:40:13-rados-testing-laggy-distro-default-smithi/7069133 --name amathuri-2022-10-16_17:40:13-rados-testing-laggy-distro-default-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} -- /home/teuthworker/archive/amathuri-2022-10-16_17:40:13-rados-testing-laggy-distro-default-smithi/7069133/orig.config.yaml 2022-10-16T17:49:58.643 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/amathuri-2022-10-16_17:40:13-rados-testing-laggy-distro-default-smithi/7069133 2022-10-16T17:49:58.644 INFO:teuthology.dispatcher.supervisor:Job PID: 18058 2022-10-16T17:49:58.645 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-10-17T05:50:52.908 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-10-17T05:50:53.065 INFO:teuthology.kill:Killing Pids: {18058} 2022-10-17T05:50:54.460 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-10-17T05:50:54.461 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-10-17T05:50:54.462 INFO:teuthology.misc:Compressing logs... 2022-10-17T05:50:54.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2022-10-17T05:50:54.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2022-10-17T05:50:54.976 INFO:teuthology.misc:Compressing logs... 2022-10-17T05:50:56.304 INFO:teuthology.misc:Compressing logs... 2022-10-17T05:56:07.824 INFO:teuthology.kill:No teuthology processes running 2022-10-17T05:56:09.453 INFO:teuthology.kill:Nuking machines: ['smithi027', 'smithi050'] 2022-10-17T05:56:10.766 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-10-17T05:56:10.767 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-10-17T05:56:10.768 INFO:teuthology.kill:2022-10-17 05:56:10,766.766 INFO:teuthology.nuke:targets: 2022-10-17T05:56:10.768 INFO:teuthology.kill: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIw6WkE/trc2jzV7Y7uNIE38gA5ZSFItoZmR/9ZBcAfOo37nO4TGy9IaJb5hMh2Q8A+rIBh5l2TOutXMQFCn1/w= 2022-10-17T05:56:10.768 INFO:teuthology.kill: smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGK0O86u32ZSVEN69BhOR4U0ghuhFP2ywZ4ij5ApUP3MU75CsEwV6gZOFfbLsNdQRyznhu8Lbm2rauRJ6mT0pLZISPZ95QhDI09nOtSsM3TBjzYR6iZUCDC/FhoGCip4XJBnkAfCEyqlFHaabdrjesLEXOSBbNAfCR+7/vNs6saLEgsg20EmLwX4I+Ku1TKiycVNAqdoB7SSPim+r39fiYF908rqbabOalXGoknKZNeBKaw+HqpT911KocMEtJBrBDrvc7ndGCVZN3bCnMETg/lsfWdsfM5kzaMHQk6+MHkt1hxZAVDMN9x19vAREWOADrTAWtC8Efy+lKP3o6WubZX5Wz37TzvY5UOScu8yeLfZwD4/ZPw3OAwbJ3gRwMJCX6+7eV0O7+akxDGNQnKDjQ7f59qNyOlqoBxzRMU5facXWmiaVcQUSafVN51jNbiRWrwSG0j4R3LF3y5Ivp9GkQOLXz4Pt+9ENsOz/M02Z8ip0snjcKaHKXwriZO9vQAF0= 2022-10-17T05:56:10.769 INFO:teuthology.kill:2022-10-17 05:56:10,768.768 INFO:teuthology.nuke:Checking targets against current locks 2022-10-17T05:56:10.835 INFO:teuthology.kill:2022-10-17 05:56:10,834.834 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-17T05:56:10.860 INFO:teuthology.kill:2022-10-17 05:56:10,860.860 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-17T05:56:10.946 INFO:teuthology.kill:2022-10-17 05:56:10,945.945 INFO:teuthology.orchestra.console:Power off smithi027 2022-10-17T05:56:10.992 INFO:teuthology.kill:2022-10-17 05:56:10,991.991 INFO:teuthology.orchestra.console:Power off smithi050 2022-10-17T05:56:19.334 INFO:teuthology.kill:2022-10-17 05:56:19,333.333 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2022-10-17T05:56:19.496 INFO:teuthology.kill:2022-10-17 05:56:19,495.495 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com 2022-10-17T05:56:23.478 INFO:teuthology.kill:2022-10-17 05:56:23,477.477 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2022-10-17T05:56:23.633 INFO:teuthology.kill:2022-10-17 05:56:23,633.633 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com 2022-10-17T05:58:24.112 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-10-17T05:58:24.444 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi027 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-10-17_02:01:03-rbd-quincy-distro-default-smithi/7069305 2022-10-17T05:58:24.444 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi050 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-10-17_02:01:03-rbd-quincy-distro-default-smithi/7069305