2021-12-15T00:06:09.911 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-15T00:06:10.082 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging 2021-12-15T00:06:10.083 INFO:teuthology.lock.ops:Updating [ubuntu@smithi149.front.sepia.ceph.com]: reset os type and version on server 2021-12-15T00:06:10.084 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2021-12-15T00:06:10.106 INFO:teuthology.lock.ops:Node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging is complete 2021-12-15T00:06:10.107 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging 2021-12-15T00:06:10.107 INFO:teuthology.lock.ops:Updating [ubuntu@smithi172.front.sepia.ceph.com]: reset os type and version on server 2021-12-15T00:06:10.108 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2021-12-15T00:06:10.131 INFO:teuthology.lock.ops:Node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging is complete 2021-12-15T00:06:10.131 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging 2021-12-15T00:06:10.132 INFO:teuthology.lock.ops:Updating [ubuntu@smithi179.front.sepia.ceph.com]: reset os type and version on server 2021-12-15T00:06:10.132 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2021-12-15T00:06:10.153 INFO:teuthology.lock.ops:Node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging is complete 2021-12-15T00:06:10.328 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of ubuntu 20.04 2021-12-15T00:06:10.347 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of ubuntu 20.04 2021-12-15T00:06:10.361 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of ubuntu 20.04 2021-12-15T00:06:10.722 INFO:teuthology.orchestra.console:Power off smithi149 2021-12-15T00:06:10.740 INFO:teuthology.orchestra.console:Power off smithi172 2021-12-15T00:06:10.761 INFO:teuthology.orchestra.console:Power off smithi179 2021-12-15T00:06:19.107 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2021-12-15T00:06:19.209 INFO:teuthology.orchestra.console:Power on smithi172 2021-12-15T00:06:23.213 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2021-12-15T00:06:23.230 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2021-12-15T00:06:23.315 INFO:teuthology.orchestra.console:Power on smithi149 2021-12-15T00:06:23.331 INFO:teuthology.orchestra.console:Power on smithi179 2021-12-15T00:06:27.573 INFO:teuthology.orchestra.console:Power on for smithi149 completed 2021-12-15T00:06:27.578 INFO:teuthology.orchestra.console:Power on for smithi172 completed 2021-12-15T00:06:27.583 INFO:teuthology.orchestra.console:Power on for smithi179 completed 2021-12-15T00:06:27.674 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish 2021-12-15T00:06:27.679 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2021-12-15T00:06:27.684 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish 2021-12-15T00:11:47.603 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-15T00:11:48.101 INFO:teuthology.provision.fog.smithi179:Node is ready 2021-12-15T00:11:48.108 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com 2021-12-15T00:11:48.115 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179 2021-12-15T00:11:48.301 INFO:teuthology.provision.fog.smithi179:Deploy complete! 2021-12-15T00:12:02.711 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-15T00:12:03.249 INFO:teuthology.provision.fog.smithi149:Node is ready 2021-12-15T00:12:03.256 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com 2021-12-15T00:12:03.264 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149 2021-12-15T00:12:03.495 INFO:teuthology.provision.fog.smithi149:Deploy complete! 2021-12-15T00:12:16.203 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-15T00:12:16.762 INFO:teuthology.provision.fog.smithi172:Node is ready 2021-12-15T00:12:16.768 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com 2021-12-15T00:12:16.775 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2021-12-15T00:12:16.975 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2021-12-15T00:12:17.193 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com 2021-12-15T00:12:17.194 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-15T00:12:17.212 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2021-12-15T00:12:17.212 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-15T00:12:17.229 INFO:teuthology.lock.ops:Checking smithi179.front.sepia.ceph.com 2021-12-15T00:12:17.229 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-15T00:12:17.247 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server 2021-12-15T00:12:17.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2021-12-15T00:12:17.431 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2021-12-15T00:12:17.479 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Ubuntu" 2021-12-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-12-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi149.stdout:ID=ubuntu 2021-12-15T00:12:17.480 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE=debian 2021-12-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-12-15T00:12:17.481 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="20.04" 2021-12-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.ubuntu.com/" 2021-12-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi149.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-12-15T00:12:17.482 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-12-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi149.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-12-15T00:12:17.483 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_CODENAME=focal 2021-12-15T00:12:17.484 INFO:teuthology.orchestra.run.smithi149.stdout:UBUNTU_CODENAME=focal 2021-12-15T00:12:17.486 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2021-12-15T00:12:17.504 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2021-12-15T00:12:17.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2021-12-15T00:12:17.674 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2021-12-15T00:12:17.724 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Ubuntu" 2021-12-15T00:12:17.725 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-12-15T00:12:17.725 INFO:teuthology.orchestra.run.smithi172.stdout:ID=ubuntu 2021-12-15T00:12:17.725 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE=debian 2021-12-15T00:12:17.725 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-12-15T00:12:17.726 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="20.04" 2021-12-15T00:12:17.726 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.ubuntu.com/" 2021-12-15T00:12:17.726 INFO:teuthology.orchestra.run.smithi172.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-12-15T00:12:17.726 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-12-15T00:12:17.727 INFO:teuthology.orchestra.run.smithi172.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-12-15T00:12:17.727 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_CODENAME=focal 2021-12-15T00:12:17.727 INFO:teuthology.orchestra.run.smithi172.stdout:UBUNTU_CODENAME=focal 2021-12-15T00:12:17.728 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2021-12-15T00:12:17.744 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: set os type and version on server 2021-12-15T00:12:17.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2021-12-15T00:12:18.001 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2021-12-15T00:12:18.008 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="Ubuntu" 2021-12-15T00:12:18.008 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-12-15T00:12:18.009 INFO:teuthology.orchestra.run.smithi179.stdout:ID=ubuntu 2021-12-15T00:12:18.009 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE=debian 2021-12-15T00:12:18.009 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-12-15T00:12:18.009 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="20.04" 2021-12-15T00:12:18.010 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://www.ubuntu.com/" 2021-12-15T00:12:18.010 INFO:teuthology.orchestra.run.smithi179.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-12-15T00:12:18.010 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-12-15T00:12:18.010 INFO:teuthology.orchestra.run.smithi179.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-12-15T00:12:18.010 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_CODENAME=focal 2021-12-15T00:12:18.011 INFO:teuthology.orchestra.run.smithi179.stdout:UBUNTU_CODENAME=focal 2021-12-15T00:12:18.011 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2021-12-15T00:12:18.106 INFO:teuthology.dispatcher.supervisor:Running job 6563144 2021-12-15T00:12:18.106 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-12-14_20:57:37-rados-wip-yuri2-testing-2021-12-14-1127-pacific-distro-default-smithi/6563144 --name yuriw-2021-12-14_20:57:37-rados-wip-yuri2-testing-2021-12-14-1127-pacific-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/yuriw-2021-12-14_20:57:37-rados-wip-yuri2-testing-2021-12-14-1127-pacific-distro-default-smithi/6563144/orig.config.yaml 2021-12-15T00:12:18.109 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-12-14_20:57:37-rados-wip-yuri2-testing-2021-12-14-1127-pacific-distro-default-smithi/6563144 2021-12-15T00:12:18.110 INFO:teuthology.dispatcher.supervisor:Job PID: 15332 2021-12-15T00:12:18.111 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-15T12:13:11.070 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-15T12:13:11.277 INFO:teuthology.kill:Killing Pids: {15332} 2021-12-15T12:13:12.473 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2021-12-15T12:13:12.473 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2021-12-15T12:13:12.474 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2021-12-15T12:13:12.474 INFO:teuthology.misc:Compressing logs... 2021-12-15T12:13:12.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2021-12-15T12:13:12.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2021-12-15T12:13:12.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2021-12-15T12:13:13.208 INFO:teuthology.misc:Compressing logs... 2021-12-15T12:13:14.412 INFO:teuthology.misc:Compressing logs... 2021-12-15T12:17:25.919 INFO:teuthology.kill:No teuthology processes running 2021-12-15T12:17:27.027 INFO:teuthology.kill:Nuking machines: ['smithi149', 'smithi172', 'smithi179'] 2021-12-15T12:17:27.936 INFO:teuthology.kill:2021-12-15 12:17:27,936.936 INFO:teuthology.nuke:targets: 2021-12-15T12:17:27.937 INFO:teuthology.kill: smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp9YHg9UqQ7Dp5PLkO6q2FW3EF2g5ZIxZ8E1jE72cI1e6W2JdBv40lx06CnEHnU0YShdjIBJcY3ReAA1IbkcgmcZZb78pW+DTrGjbOmUGualHu5zv7vhACPoJ3BnM8a+f1nfPUcELXrsA2JbBMMp07OwRp/pwBqpOBrOYRZalbtJ0y33Nl8uUoHH/4NyO1I52QSblOLpV7A6NQ7QijJFHUogE7PSmRxhHacq8epJbQ8R5B7Vmv0Cmq0AtasA0dal1FiAvfF7MMLsKRwfsX2pOof5XCp1vruPjxHby0/v6sSWstVmWe4fJp8veWkBZiHCtjP1w/ow2L8uBiTSI7IDW1+5VN5/Eo/paBGgVrMLNiK6hBOdSa8patKlpT4CnBuilmPVqaJmbQ/HxFmutKwzJmot34N5ZU2GL6gK42fpxAgaSknHVw3EMaLi7vzwlGZ0Cq9ufvG7UbveIovNCYH/k+TBWlSXie4TUm8I+4KPDKeE/5BNVJvK38PKFxjaYYyos= 2021-12-15T12:17:27.937 INFO:teuthology.kill: smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDzt46fM9jdDwxtvD0MQhKBh8hpEbJ1J2pmUGoKhDssLwnqSmOQ0/384be43ikNEfl4OzU2sd7qqZ/BHpOmCf5/ltlSOwMJb36tAraL5uFL4OgbxcOjVDmqh2QlVHrDEI36YsNtioqPYgUk9T/T0c48tcuFVW+1kMPpGQU4ZSsz15IuiJKHvbxdEngT3NiSTf3Pj5x1RYugavKkumW43wJNGYgXaAnvJrU/5NoGYQgqur7qeDAGOXk9Ag6aNmMptqE/lvxpu6K7/YALAP80COYVYB505RadBcCRaM2xezzTEEF+DKhZ0C1CcwNoFRk+OqzCC89nmmB/jCKcFzVgEUALX33IH1WVlVFM33pnjjhG12f9A31YMq8WJFtw4KDFQoiKaTsSNE2E9onigFgDAaKdgNQ9UBQ4WyCbeArBBGOSetWtmVZU1DQJANDoakreER9rZPrXkfyPKqmcXokAuFAmN/DSPtJylHlhm9l0h1mayhvGZnNnzrQiPLN8G7dcOos= 2021-12-15T12:17:27.938 INFO:teuthology.kill: smithi179.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfCd8/jxP/Ajz2zpUq7XaIL2ZfXlwIlfthuCQB5hTKtNvZ1pIhc5wWa6SQ+2dFEkljUM2/w88j1GaF1V6alSfC1DNzF+oyUaUYBaROpLQ0Mw1lzSqbRs1VwNYBsbdcqgd3ly3GnPpFfltDHwKl1yI/cB73HJT4SszZqulARCvKadYF/MJqtMV0FNvilroPP0Y0s5dKlrehxOFDj+6mERl08Zvef0avWMthcUjUnvrNSsKJe8cvCeGQtuZQ9I68g9/O0SjyeKTDw911T/6M/LR7YlQqrIrY6fVxe0OIE58Q7JCDheRyAZCn3EY+J9nihs1bZyPnCyB4/Xsgbqsk/5RKClXj63M81LMDbsoL/fSrjPa9Xt+Ow6I3TfoQ8WkFLzhqvZWQr/G0xqKfBy0kVg/H9ZfDp5690znnE0J/M7g08ww+v1ppxtCy7YKNJ8AqdaYjXVBXUVgXXVb/9kgYt/TzIlTxITezM6InrOvAMawwfLYfze4mxVxJmfE3hhpltyc= 2021-12-15T12:17:27.967 INFO:teuthology.kill:2021-12-15 12:17:27,966.966 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-15T12:17:27.974 INFO:teuthology.kill:2021-12-15 12:17:27,973.973 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-15T12:17:27.976 INFO:teuthology.kill:2021-12-15 12:17:27,976.976 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-15T12:17:28.070 INFO:teuthology.kill:2021-12-15 12:17:28,070.070 INFO:teuthology.orchestra.console:Power off smithi172 2021-12-15T12:17:28.086 INFO:teuthology.kill:2021-12-15 12:17:28,085.085 INFO:teuthology.orchestra.console:Power off smithi149 2021-12-15T12:17:28.097 INFO:teuthology.kill:2021-12-15 12:17:28,097.097 INFO:teuthology.orchestra.console:Power off smithi179 2021-12-15T12:17:36.458 INFO:teuthology.kill:2021-12-15 12:17:36,457.457 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2021-12-15T12:17:36.465 INFO:teuthology.kill:2021-12-15 12:17:36,465.465 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2021-12-15T12:17:36.605 INFO:teuthology.kill:2021-12-15 12:17:36,604.604 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com 2021-12-15T12:17:36.608 INFO:teuthology.kill:2021-12-15 12:17:36,608.608 INFO:teuthology.lock.ops:unlocked: smithi179.front.sepia.ceph.com 2021-12-15T12:17:40.567 INFO:teuthology.kill:2021-12-15 12:17:40,567.567 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2021-12-15T12:17:40.714 INFO:teuthology.kill:2021-12-15 12:17:40,713.713 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com 2021-12-15T12:19:40.955 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-15T12:19:41.098 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi149 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-15_12:17:03-powercycle-pacific-distro-default-smithi/6563930 2021-12-15T12:19:41.098 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi172 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-15_12:17:03-powercycle-pacific-distro-default-smithi/6563930 2021-12-15T12:19:41.098 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi179 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-15_12:17:03-powercycle-pacific-distro-default-smithi/6563930