2022-01-26T04:56:07.351 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-26T04:56:07.511 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi096.front.sepia.ceph.com' reimaging 2022-01-26T04:56:07.512 INFO:teuthology.lock.ops:Updating [ubuntu@smithi096.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T04:56:07.513 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2022-01-26T04:56:07.534 INFO:teuthology.lock.ops:Node 'ubuntu@smithi096.front.sepia.ceph.com' reimaging is complete 2022-01-26T04:56:07.534 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging 2022-01-26T04:56:07.535 INFO:teuthology.lock.ops:Updating [ubuntu@smithi150.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T04:56:07.535 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2022-01-26T04:56:07.556 INFO:teuthology.lock.ops:Node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging is complete 2022-01-26T04:56:07.557 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi156.front.sepia.ceph.com' reimaging 2022-01-26T04:56:07.558 INFO:teuthology.lock.ops:Updating [ubuntu@smithi156.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T04:56:07.558 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2022-01-26T04:56:07.580 INFO:teuthology.lock.ops:Node 'ubuntu@smithi156.front.sepia.ceph.com' reimaging is complete 2022-01-26T04:56:07.741 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of ubuntu 20.04 2022-01-26T04:56:07.763 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of ubuntu 20.04 2022-01-26T04:56:07.774 INFO:teuthology.provision.fog.smithi156:Scheduling deploy of ubuntu 20.04 2022-01-26T04:56:08.017 INFO:teuthology.orchestra.console:Power off smithi096 2022-01-26T04:56:08.058 INFO:teuthology.orchestra.console:Power off smithi150 2022-01-26T04:56:08.090 INFO:teuthology.orchestra.console:Power off smithi156 2022-01-26T04:56:16.454 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2022-01-26T04:56:16.555 INFO:teuthology.orchestra.console:Power on smithi156 2022-01-26T04:56:20.509 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2022-01-26T04:56:20.529 INFO:teuthology.orchestra.console:Power off for smithi150 completed 2022-01-26T04:56:20.611 INFO:teuthology.orchestra.console:Power on smithi096 2022-01-26T04:56:20.631 INFO:teuthology.orchestra.console:Power on smithi150 2022-01-26T04:56:24.878 INFO:teuthology.orchestra.console:Power on for smithi096 completed 2022-01-26T04:56:24.885 INFO:teuthology.orchestra.console:Power on for smithi150 completed 2022-01-26T04:56:24.918 INFO:teuthology.orchestra.console:Power on for smithi156 completed 2022-01-26T04:56:24.980 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish 2022-01-26T04:56:24.986 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish 2022-01-26T04:56:25.019 INFO:teuthology.provision.fog.smithi156:Waiting for deploy to finish 2022-01-26T05:01:34.527 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T05:01:35.077 INFO:teuthology.provision.fog.smithi096:Node is ready 2022-01-26T05:01:35.084 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com 2022-01-26T05:01:35.091 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096 2022-01-26T05:01:35.318 INFO:teuthology.provision.fog.smithi096:Deploy complete! 2022-01-26T05:01:44.621 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T05:01:45.268 INFO:teuthology.provision.fog.smithi156:Node is ready 2022-01-26T05:01:45.275 INFO:teuthology.orchestra.run.smithi156.stdout:smithi156.front.sepia.ceph.com 2022-01-26T05:01:45.282 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156 smithi156.front.sepia.ceph.com smithi156 2022-01-26T05:01:45.464 INFO:teuthology.provision.fog.smithi156:Deploy complete! 2022-01-26T05:01:56.558 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T05:01:57.107 INFO:teuthology.provision.fog.smithi150:Node is ready 2022-01-26T05:01:57.114 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com 2022-01-26T05:01:57.121 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150 2022-01-26T05:01:57.434 INFO:teuthology.provision.fog.smithi150:Deploy complete! 2022-01-26T05:01:57.677 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com 2022-01-26T05:01:57.678 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T05:01:57.700 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com 2022-01-26T05:01:57.701 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T05:01:57.719 INFO:teuthology.lock.ops:Checking smithi156.front.sepia.ceph.com 2022-01-26T05:01:57.720 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T05:01:57.745 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server 2022-01-26T05:01:57.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2022-01-26T05:01:58.010 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2022-01-26T05:01:58.017 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2022-01-26T05:01:58.018 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-26T05:01:58.018 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2022-01-26T05:01:58.018 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2022-01-26T05:01:58.018 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-26T05:01:58.019 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="20.04" 2022-01-26T05:01:58.019 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-26T05:01:58.019 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-26T05:01:58.019 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-26T05:01:58.020 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-26T05:01:58.020 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=focal 2022-01-26T05:01:58.020 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=focal 2022-01-26T05:01:58.022 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2022-01-26T05:01:58.042 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server 2022-01-26T05:01:58.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2022-01-26T05:01:58.222 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2022-01-26T05:01:58.230 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Ubuntu" 2022-01-26T05:01:58.230 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-26T05:01:58.231 INFO:teuthology.orchestra.run.smithi150.stdout:ID=ubuntu 2022-01-26T05:01:58.231 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE=debian 2022-01-26T05:01:58.231 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-26T05:01:58.232 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="20.04" 2022-01-26T05:01:58.232 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-26T05:01:58.232 INFO:teuthology.orchestra.run.smithi150.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-26T05:01:58.232 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-26T05:01:58.233 INFO:teuthology.orchestra.run.smithi150.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-26T05:01:58.233 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_CODENAME=focal 2022-01-26T05:01:58.233 INFO:teuthology.orchestra.run.smithi150.stdout:UBUNTU_CODENAME=focal 2022-01-26T05:01:58.236 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2022-01-26T05:01:58.256 INFO:teuthology.lock.ops:Updating [smithi156.front.sepia.ceph.com]: set os type and version on server 2022-01-26T05:01:58.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2022-01-26T05:01:58.439 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2022-01-26T05:01:58.445 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Ubuntu" 2022-01-26T05:01:58.446 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-26T05:01:58.446 INFO:teuthology.orchestra.run.smithi156.stdout:ID=ubuntu 2022-01-26T05:01:58.446 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE=debian 2022-01-26T05:01:58.447 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-26T05:01:58.447 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="20.04" 2022-01-26T05:01:58.447 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-26T05:01:58.447 INFO:teuthology.orchestra.run.smithi156.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-26T05:01:58.448 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-26T05:01:58.448 INFO:teuthology.orchestra.run.smithi156.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-26T05:01:58.448 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_CODENAME=focal 2022-01-26T05:01:58.448 INFO:teuthology.orchestra.run.smithi156.stdout:UBUNTU_CODENAME=focal 2022-01-26T05:01:58.449 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2022-01-26T05:01:58.544 INFO:teuthology.dispatcher.supervisor:Running job 6638782 2022-01-26T05:01:58.545 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-01-25_03:15:03-fs-master-distro-default-smithi/6638782 --name teuthology-2022-01-25_03:15:03-fs-master-distro-default-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} -- /home/teuthworker/archive/teuthology-2022-01-25_03:15:03-fs-master-distro-default-smithi/6638782/orig.config.yaml 2022-01-26T05:01:58.548 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-01-25_03:15:03-fs-master-distro-default-smithi/6638782 2022-01-26T05:01:58.549 INFO:teuthology.dispatcher.supervisor:Job PID: 25984 2022-01-26T05:01:58.550 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-26T11:32:28.439 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-26T11:32:28.621 INFO:teuthology.kill:Killing Pids: {25984} 2022-01-26T11:32:29.797 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2022-01-26T11:32:29.798 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2022-01-26T11:32:29.798 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2022-01-26T11:32:29.799 INFO:teuthology.misc:Compressing logs... 2022-01-26T11:32:29.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2022-01-26T11:32:30.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2022-01-26T11:32:30.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2022-01-26T11:32:30.684 INFO:teuthology.misc:Compressing logs... 2022-01-26T11:32:30.878 INFO:teuthology.misc:Compressing logs... 2022-01-26T11:38:09.076 INFO:teuthology.kill:No teuthology processes running 2022-01-26T11:38:10.201 INFO:teuthology.kill:Nuking machines: ['smithi096', 'smithi150', 'smithi156'] 2022-01-26T11:38:11.218 INFO:teuthology.kill:2022-01-26 11:38:11,218.218 INFO:teuthology.nuke:targets: 2022-01-26T11:38:11.219 INFO:teuthology.kill: smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRDISa+DJRDwdZ7HmXDQLshEiipJPW0Wa9breoVLcHF6Y2H5g2RyYKl2kDxjV/jbIFRjc8pEpLOOZkOSxRsuXziyj+p9/FZNPnS/SBLyU8fv9EHv9jy9sAEhwE469YxtHlzAh2TiWwVNR2xZntEscW/O0f/HxxHYfyUKY1KNLwpQwiN30w1+9tlRB4GFJCWVE0+83zJUJvu0I6neUhHtV8uPNeR+sMcCv0oQJJxbBMAlUP4vnmezlnBBT4OrAnPaKD5dPQQmZILnc10GDHIvJml2vOMg/abr4ChWL1ySjc7da7fD2c055ddbwXm4sDy4TWiymg7bVvwptXPJJOBVfVyNvGbGJ9wwNmwX+vBPxRS5EhZScDANV1k1NboMtEb4AuRjHZF7Hkd7Mr+xowAwBxolVjJSJdB2U+C5Nmf+o+PRzf5B8CBCRIWUH81ci8ZKt33UDYkb4GWdxRXtEBUKVq12iWBJ5F6+kQjONW46Y2mK64FVFAmzPUxXCKMdS6kSE= 2022-01-26T11:38:11.219 INFO:teuthology.kill: smithi150.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRwrI+olffhdxWLrRm5sTDj2NMPTXu+cG1JhdZBvFvI8iuOnofEFS6IY/vH0m/5qfLTkveA8U9i9i9r3PR94gh9sNxn+fk6H9h6Dw7+tUF95jqp8tNaFEbVCkyT4ovoU+uUN+JzLaS5KQpOZQCiSYL0gpAj8W8ePGPQvOXG7PYgW/5WDPppDJZVjmVbuN/f3zEk52A1DtcDFy5cECpTnHNJa4tuX7FI0lFsZoOZwR5Z/P9NJr/pI+ieSvRzH2pAs8U3JyAclOvwfLnPVqYxg4DCrdqs1yQk0vhyFW+Cym9vfB2+bgAcGvFgfaWarDpRAPfDFAPgy1m6ix3bdk9Zbmj/sLHAnj0z4ceZVxyoIiDOHVNswVP0PiZnQFjLHE9PHAjjOOepnXTpAp10pUtRtG199OPhtClhnxAYcI2OCxD4Y5OktYXiz70fLK0fIkVSmJVp4vTR865SpUfNhpKMK0TjKyyszyU3TodxTSGhcRSd4hqAXJV5HLxnyoIBGEldUc= 2022-01-26T11:38:11.219 INFO:teuthology.kill: smithi156.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC56m3BY4LC2VIciyxfqci58G/uJCkge3YOyoZ1G0kE/Gz8hVs7AezhKe9VUFPgw7IhjKnTdT9wiSF49Fq8zqLx87t4S11caRasdUMy5Sc3MH60fKs9ca2FXlPBqFwO+oaOKeve0yC6IHbMgVLFi49fdN/QkG/HcvSzrwqP6gAGwIiwg9E2mT7YLzVJfWrYHWyG5YPjm8Yw//Ykza/jKqjycln3b15F8Nhf21/L+oOILqhtq2oKHe1NW2BYrdxR6it2hRERyTENf5RmTIW4zAtBigYWL1IiMi+jdVtzB5O1aISW/VzhfKD+6w1CQWoZ0Oz9ZoJu73WkYrVXp+ytITNd7LL3hbhMTXp80hvdYyilw4LeTQT/m6AznOLPg9Jq8sYcBIAkaG9bfW2zvSOZddVwgm7a1ey1MvoGCS9i6ASJkRF2sySjt5Xx3MNrumENdyw9nAUTBoeDT4M5tOhUEgX7T41axrE8M0vqldlg+kfoCfieMxCr/H2cp4untcAszpU= 2022-01-26T11:38:11.248 INFO:teuthology.kill:2022-01-26 11:38:11,248.248 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T11:38:11.252 INFO:teuthology.kill:2022-01-26 11:38:11,252.252 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T11:38:11.254 INFO:teuthology.kill:2022-01-26 11:38:11,253.253 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T11:38:11.333 INFO:teuthology.kill:2022-01-26 11:38:11,333.333 INFO:teuthology.orchestra.console:Power off smithi156 2022-01-26T11:38:11.347 INFO:teuthology.kill:2022-01-26 11:38:11,346.346 INFO:teuthology.orchestra.console:Power off smithi150 2022-01-26T11:38:11.361 INFO:teuthology.kill:2022-01-26 11:38:11,361.361 INFO:teuthology.orchestra.console:Power off smithi096 2022-01-26T11:38:19.716 INFO:teuthology.kill:2022-01-26 11:38:19,716.716 INFO:teuthology.orchestra.console:Power off for smithi150 completed 2022-01-26T11:38:19.728 INFO:teuthology.kill:2022-01-26 11:38:19,728.728 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2022-01-26T11:38:19.865 INFO:teuthology.kill:2022-01-26 11:38:19,864.864 INFO:teuthology.lock.ops:unlocked: smithi150.front.sepia.ceph.com 2022-01-26T11:38:19.870 INFO:teuthology.kill:2022-01-26 11:38:19,870.870 INFO:teuthology.lock.ops:unlocked: smithi096.front.sepia.ceph.com 2022-01-26T11:38:23.832 INFO:teuthology.kill:2022-01-26 11:38:23,832.832 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2022-01-26T11:38:23.982 INFO:teuthology.kill:2022-01-26 11:38:23,982.982 INFO:teuthology.lock.ops:unlocked: smithi156.front.sepia.ceph.com 2022-01-26T11:40:24.315 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-26T11:40:24.459 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi096 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-25_05:15:02-krbd-octopus-testing-default-smithi/6639770 2022-01-26T11:40:24.459 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi150 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-25_05:15:02-krbd-octopus-testing-default-smithi/6639770 2022-01-26T11:40:24.460 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi156 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-25_05:15:02-krbd-octopus-testing-default-smithi/6639770