2022-02-12T00:10:26.970 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-12T00:10:27.137 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging 2022-02-12T00:10:27.138 INFO:teuthology.lock.ops:Updating [ubuntu@smithi068.front.sepia.ceph.com]: reset os type and version on server 2022-02-12T00:10:27.139 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server 2022-02-12T00:10:27.157 INFO:teuthology.lock.ops:Node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging is complete 2022-02-12T00:10:27.158 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi124.front.sepia.ceph.com' reimaging 2022-02-12T00:10:27.158 INFO:teuthology.lock.ops:Updating [ubuntu@smithi124.front.sepia.ceph.com]: reset os type and version on server 2022-02-12T00:10:27.158 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2022-02-12T00:10:27.177 INFO:teuthology.lock.ops:Node 'ubuntu@smithi124.front.sepia.ceph.com' reimaging is complete 2022-02-12T00:10:27.177 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi142.front.sepia.ceph.com' reimaging 2022-02-12T00:10:27.178 INFO:teuthology.lock.ops:Updating [ubuntu@smithi142.front.sepia.ceph.com]: reset os type and version on server 2022-02-12T00:10:27.178 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2022-02-12T00:10:27.195 INFO:teuthology.lock.ops:Node 'ubuntu@smithi142.front.sepia.ceph.com' reimaging is complete 2022-02-12T00:10:27.376 INFO:teuthology.provision.fog.smithi068:Scheduling deploy of ubuntu 20.04 2022-02-12T00:10:27.400 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of ubuntu 20.04 2022-02-12T00:10:27.444 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of ubuntu 20.04 2022-02-12T00:10:27.747 INFO:teuthology.orchestra.console:Power off smithi068 2022-02-12T00:10:27.791 INFO:teuthology.orchestra.console:Power off smithi124 2022-02-12T00:10:27.823 INFO:teuthology.orchestra.console:Power off smithi142 2022-02-12T00:10:27.900 INFO:teuthology.orchestra.console:Power off for smithi068 completed 2022-02-12T00:10:27.964 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2022-02-12T00:10:28.002 INFO:teuthology.orchestra.console:Power on smithi068 2022-02-12T00:10:28.066 INFO:teuthology.orchestra.console:Power on smithi142 2022-02-12T00:10:36.361 INFO:teuthology.orchestra.console:Power on for smithi068 completed 2022-02-12T00:10:36.428 INFO:teuthology.orchestra.console:Power on for smithi142 completed 2022-02-12T00:10:36.463 INFO:teuthology.provision.fog.smithi068:Waiting for deploy to finish 2022-02-12T00:10:36.528 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish 2022-02-12T00:10:40.271 INFO:teuthology.orchestra.console:Power off for smithi124 completed 2022-02-12T00:10:40.372 INFO:teuthology.orchestra.console:Power on smithi124 2022-02-12T00:10:44.623 INFO:teuthology.orchestra.console:Power on for smithi124 completed 2022-02-12T00:10:44.724 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish 2022-02-12T00:15:28.764 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-12T00:15:29.273 INFO:teuthology.provision.fog.smithi142:Node is ready 2022-02-12T00:15:29.279 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142.front.sepia.ceph.com 2022-02-12T00:15:29.286 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142 smithi142.front.sepia.ceph.com smithi142 2022-02-12T00:15:29.468 INFO:teuthology.provision.fog.smithi142:Deploy complete! 2022-02-12T00:15:46.491 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-12T00:15:46.945 INFO:teuthology.provision.fog.smithi068:Node is ready 2022-02-12T00:15:46.952 INFO:teuthology.orchestra.run.smithi068.stdout:smithi068.front.sepia.ceph.com 2022-02-12T00:15:47.002 INFO:teuthology.orchestra.run.smithi068.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068 2022-02-12T00:15:47.237 INFO:teuthology.provision.fog.smithi068:Deploy complete! 2022-02-12T00:15:51.703 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-12T00:15:52.220 INFO:teuthology.provision.fog.smithi124:Node is ready 2022-02-12T00:15:52.226 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com 2022-02-12T00:15:52.233 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124 2022-02-12T00:15:52.438 INFO:teuthology.provision.fog.smithi124:Deploy complete! 2022-02-12T00:15:52.673 INFO:teuthology.lock.ops:Checking smithi068.front.sepia.ceph.com 2022-02-12T00:15:52.674 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-12T00:15:52.697 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com 2022-02-12T00:15:52.698 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-12T00:15:52.714 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com 2022-02-12T00:15:52.715 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-12T00:15:52.736 INFO:teuthology.lock.ops:Updating [smithi068.front.sepia.ceph.com]: set os type and version on server 2022-02-12T00:15:52.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com' 2022-02-12T00:15:52.966 INFO:teuthology.orchestra.run.smithi068.stdout:x86_64 2022-02-12T00:15:53.025 INFO:teuthology.orchestra.run.smithi068.stdout:NAME="Ubuntu" 2022-02-12T00:15:53.026 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-12T00:15:53.027 INFO:teuthology.orchestra.run.smithi068.stdout:ID=ubuntu 2022-02-12T00:15:53.027 INFO:teuthology.orchestra.run.smithi068.stdout:ID_LIKE=debian 2022-02-12T00:15:53.028 INFO:teuthology.orchestra.run.smithi068.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-12T00:15:53.029 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_ID="20.04" 2022-02-12T00:15:53.029 INFO:teuthology.orchestra.run.smithi068.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-12T00:15:53.031 INFO:teuthology.orchestra.run.smithi068.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-12T00:15:53.031 INFO:teuthology.orchestra.run.smithi068.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-12T00:15:53.031 INFO:teuthology.orchestra.run.smithi068.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-12T00:15:53.031 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_CODENAME=focal 2022-02-12T00:15:53.032 INFO:teuthology.orchestra.run.smithi068.stdout:UBUNTU_CODENAME=focal 2022-02-12T00:15:53.033 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server 2022-02-12T00:15:53.058 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server 2022-02-12T00:15:53.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2022-02-12T00:15:53.300 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2022-02-12T00:15:53.308 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Ubuntu" 2022-02-12T00:15:53.308 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-12T00:15:53.309 INFO:teuthology.orchestra.run.smithi124.stdout:ID=ubuntu 2022-02-12T00:15:53.309 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE=debian 2022-02-12T00:15:53.309 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-12T00:15:53.310 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="20.04" 2022-02-12T00:15:53.310 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-12T00:15:53.310 INFO:teuthology.orchestra.run.smithi124.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-12T00:15:53.311 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-12T00:15:53.311 INFO:teuthology.orchestra.run.smithi124.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-12T00:15:53.311 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_CODENAME=focal 2022-02-12T00:15:53.312 INFO:teuthology.orchestra.run.smithi124.stdout:UBUNTU_CODENAME=focal 2022-02-12T00:15:53.312 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2022-02-12T00:15:53.334 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server 2022-02-12T00:15:53.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2022-02-12T00:15:53.531 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2022-02-12T00:15:53.540 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Ubuntu" 2022-02-12T00:15:53.540 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-12T00:15:53.541 INFO:teuthology.orchestra.run.smithi142.stdout:ID=ubuntu 2022-02-12T00:15:53.541 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE=debian 2022-02-12T00:15:53.541 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-12T00:15:53.542 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="20.04" 2022-02-12T00:15:53.543 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-12T00:15:53.543 INFO:teuthology.orchestra.run.smithi142.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-12T00:15:53.544 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-12T00:15:53.544 INFO:teuthology.orchestra.run.smithi142.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-12T00:15:53.544 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_CODENAME=focal 2022-02-12T00:15:53.545 INFO:teuthology.orchestra.run.smithi142.stdout:UBUNTU_CODENAME=focal 2022-02-12T00:15:53.545 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2022-02-12T00:15:53.667 INFO:teuthology.dispatcher.supervisor:Running job 6677418 2022-02-12T00:15:53.667 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-02-11_19:41:29-rados-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677418 --name yuriw-2022-02-11_19:41:29-rados-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/yuriw-2022-02-11_19:41:29-rados-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677418/orig.config.yaml 2022-02-12T00:15:53.670 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-02-11_19:41:29-rados-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677418 2022-02-12T00:15:53.671 INFO:teuthology.dispatcher.supervisor:Job PID: 19273 2022-02-12T00:15:53.672 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-12T06:46:24.100 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-12T06:46:24.325 INFO:teuthology.kill:Killing Pids: {19273} 2022-02-12T06:46:25.558 INFO:teuthology.task.internal:roles: ubuntu@smithi068.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2022-02-12T06:46:25.559 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2022-02-12T06:46:25.559 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2022-02-12T06:46:25.559 INFO:teuthology.misc:Compressing logs... 2022-02-12T06:46:25.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com' 2022-02-12T06:46:25.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2022-02-12T06:46:25.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2022-02-12T06:46:26.259 INFO:teuthology.misc:Compressing logs... 2022-02-12T06:46:26.959 INFO:teuthology.misc:Compressing logs... 2022-02-12T06:49:59.554 INFO:teuthology.kill:No teuthology processes running 2022-02-12T06:50:00.760 INFO:teuthology.kill:Nuking machines: ['smithi068', 'smithi124', 'smithi142'] 2022-02-12T06:50:01.761 INFO:teuthology.kill:2022-02-12 06:50:01,761.761 INFO:teuthology.nuke:targets: 2022-02-12T06:50:01.763 INFO:teuthology.kill: smithi068.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2NwOe+IjSwUab+BpklNMMajsPM3ZB4uWgpAvv0/xM7PMx3nNfMpQk5+nCQKOteGWqO6EEenIFINlX9IxmsD4SyVfrnKHu+0jeBVxk9SAg009egeTuCfU+k99DQ7QFzgJ4vHzyk88zqEqeWda7knHPBamayt9lnJo3ur6ZDm6dRtDkjERQzOYXek6g/pTp8+FPFoQhmfdzCHc9RDNNk8JyiUnYokOAMduzO4VgLUGfb2f66iDLvz1SYrHeeu1amgbp344JL+UtYcLYyRWqPWLr1DuVBvK8y3ZiWIdTDe3iL5EqmlHJxXbecsUpp2QZ24Rw2S5Im2u0zGXCkTI8ksI0Hz6mi3rb/3fx3CRnSc3/FzcTZJ5ohNZU9pQ/wsKm5dxhmzIuohWsGQqCa7NBEAScSUVawb9YTCPGfFPouCyfEicL2t3wyyGiqdbJ6FG8xOauM5ls6xYVK7KmG78sqx40/jfJ7TT9V6I8rcB8RqutY8lKThyKWa5v56sXYQyOies= 2022-02-12T06:50:01.763 INFO:teuthology.kill: smithi124.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkkkay1hOOJ9ns/EigtFQJYKhUPcbYGahDbAqxIrUzfLYUgJqt14pZdUTUrpf9YyVPjpXtDo21eZIO4Dg6FMFLwBTKxhxbTMoSSDXp5iRjSnpEs+Yab9IrepJaI/oCltzlUGLpy3P3mtM4OzJgrNWSgk5Ho/5khoZ6PRHVdS266Cg4Ycn+GdLpWUTCb0LqgXMZDnxvlu6P0FZsSGnzSFsKfa1a/Xy6EBmKxRnDPPLTySFRSs1Gp7rH5YcZ4UgkqUSyITXuKMxsGjkBuhiGXhW8ri4/gjcC7ZO/2oGVEQmO1SzspqkNVZ9RN4SEutylBzIoE1X2KCvZh+tt91rxvrwVPxmhNphfv6IjBMtjyRJLyACba6mpMqrQYVM2RghyMVhDuD5HffxwljPpAoIod/KvxTan65T09tZFLbzczaGu6G7AAc8bhEK4U7JFBQTQzsaiqqT9q+tXEpTeSQZ1zZ5lkGmnNXAFJQWj1AxMepiBlc+r2/Dxe1pt0IQd5LJcuKk= 2022-02-12T06:50:01.764 INFO:teuthology.kill: smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAm3WST9sMuSiTx7B+mZ0Us46/tIvRoQ6DAgTJb2pXoaHxfCBARurFuFbMzgkChYLyu9ue4SQ+Qcwi2ZLD8wsvjI/QXcGY+DdhCZbka2R3+NzPOgxfF/wAx4/KVbphHzTtgj/Z7PRgMHMEgCMLY4iPWK6Oc2KGIgIQb9c7zTOrpufjIVNlC8pjyJyEpjUah+c2t0Eqx1xS+qKQr88EVawB5p7sx7qzPUh9US5m03nEwe5yXx1xgjVyDijIM9We6hwouX3Tq7qwVpAWCNmQ8e4UqDTx9B6aJVMmzb2jEEmJ2oWC5kSjKMSGSplOBrqCcLbtuT1m+++E58ICmWYYdZRwI/6Pjz4BuPI9nj6oJ8r2rUicj39ulP0LE0KqZiqa3xAX6SzYGqVoQqbkyoFRP0uMs/mJSsrfDantd4nyrm0EoNAiXAUFx0TxnejHoTEM+mm+jnD0JPmt4LZPjJ7lWwgQcrJRP3jZTQGqGWJ8qyZdeqAxXQCVoDxvTZr8bmVPNlk= 2022-02-12T06:50:01.790 INFO:teuthology.kill:2022-02-12 06:50:01,790.790 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-12T06:50:01.794 INFO:teuthology.kill:2022-02-12 06:50:01,794.794 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-12T06:50:01.796 INFO:teuthology.kill:2022-02-12 06:50:01,796.796 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-12T06:50:01.883 INFO:teuthology.kill:2022-02-12 06:50:01,882.882 INFO:teuthology.orchestra.console:Power off smithi142 2022-02-12T06:50:01.903 INFO:teuthology.kill:2022-02-12 06:50:01,902.902 INFO:teuthology.orchestra.console:Power off smithi124 2022-02-12T06:50:01.916 INFO:teuthology.kill:2022-02-12 06:50:01,915.915 INFO:teuthology.orchestra.console:Power off smithi068 2022-02-12T06:50:14.393 INFO:teuthology.kill:2022-02-12 06:50:14,393.393 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2022-02-12T06:50:14.398 INFO:teuthology.kill:2022-02-12 06:50:14,398.398 INFO:teuthology.orchestra.console:Power off for smithi124 completed 2022-02-12T06:50:14.404 INFO:teuthology.kill:2022-02-12 06:50:14,404.404 INFO:teuthology.orchestra.console:Power off for smithi068 completed 2022-02-12T06:50:14.544 INFO:teuthology.kill:2022-02-12 06:50:14,543.543 INFO:teuthology.lock.ops:unlocked: smithi124.front.sepia.ceph.com 2022-02-12T06:50:14.557 INFO:teuthology.kill:2022-02-12 06:50:14,557.557 INFO:teuthology.lock.ops:unlocked: smithi142.front.sepia.ceph.com 2022-02-12T06:50:14.561 INFO:teuthology.kill:2022-02-12 06:50:14,561.561 INFO:teuthology.lock.ops:unlocked: smithi068.front.sepia.ceph.com 2022-02-12T06:52:14.881 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-12T06:52:15.004 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi068 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-11_04:10:02-fs-octopus-distro-default-smithi/6675654 2022-02-12T06:52:15.004 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi142 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-11_04:10:02-fs-octopus-distro-default-smithi/6675654