2022-02-07T23:41:20.973 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-07T23:41:21.091 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi006.front.sepia.ceph.com' reimaging 2022-02-07T23:41:21.092 INFO:teuthology.lock.ops:Updating [ubuntu@smithi006.front.sepia.ceph.com]: reset os type and version on server 2022-02-07T23:41:21.093 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2022-02-07T23:41:21.111 INFO:teuthology.lock.ops:Node 'ubuntu@smithi006.front.sepia.ceph.com' reimaging is complete 2022-02-07T23:41:21.112 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi018.front.sepia.ceph.com' reimaging 2022-02-07T23:41:21.112 INFO:teuthology.lock.ops:Updating [ubuntu@smithi018.front.sepia.ceph.com]: reset os type and version on server 2022-02-07T23:41:21.113 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2022-02-07T23:41:21.131 INFO:teuthology.lock.ops:Node 'ubuntu@smithi018.front.sepia.ceph.com' reimaging is complete 2022-02-07T23:41:21.371 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of ubuntu 20.04 2022-02-07T23:41:21.418 INFO:teuthology.provision.fog.smithi018:Scheduling deploy of ubuntu 20.04 2022-02-07T23:41:21.732 INFO:teuthology.orchestra.console:Power off smithi006 2022-02-07T23:41:21.793 INFO:teuthology.orchestra.console:Power off smithi018 2022-02-07T23:41:30.104 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2022-02-07T23:41:30.156 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2022-02-07T23:41:30.206 INFO:teuthology.orchestra.console:Power on smithi006 2022-02-07T23:41:30.257 INFO:teuthology.orchestra.console:Power on smithi018 2022-02-07T23:41:38.571 INFO:teuthology.orchestra.console:Power on for smithi006 completed 2022-02-07T23:41:38.673 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish 2022-02-07T23:41:38.678 INFO:teuthology.orchestra.console:Power on for smithi018 completed 2022-02-07T23:41:38.779 INFO:teuthology.provision.fog.smithi018:Waiting for deploy to finish 2022-02-07T23:46:16.870 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-07T23:46:16.881 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-07T23:46:17.445 INFO:teuthology.provision.fog.smithi006:Node is ready 2022-02-07T23:46:17.452 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com 2022-02-07T23:46:17.462 INFO:teuthology.provision.fog.smithi018:Node is ready 2022-02-07T23:46:17.470 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018.front.sepia.ceph.com 2022-02-07T23:46:17.502 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006 2022-02-07T23:46:17.520 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18 smithi018.front.sepia.ceph.com smithi018 2022-02-07T23:46:17.692 INFO:teuthology.provision.fog.smithi006:Deploy complete! 2022-02-07T23:46:17.698 INFO:teuthology.provision.fog.smithi018:Deploy complete! 2022-02-07T23:46:17.882 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com 2022-02-07T23:46:17.883 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-07T23:46:17.903 INFO:teuthology.lock.ops:Checking smithi018.front.sepia.ceph.com 2022-02-07T23:46:17.903 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-07T23:46:17.926 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server 2022-02-07T23:46:17.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2022-02-07T23:46:18.153 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2022-02-07T23:46:18.160 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Ubuntu" 2022-02-07T23:46:18.161 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-07T23:46:18.161 INFO:teuthology.orchestra.run.smithi006.stdout:ID=ubuntu 2022-02-07T23:46:18.161 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE=debian 2022-02-07T23:46:18.162 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-07T23:46:18.162 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="20.04" 2022-02-07T23:46:18.162 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-07T23:46:18.162 INFO:teuthology.orchestra.run.smithi006.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-07T23:46:18.163 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-07T23:46:18.163 INFO:teuthology.orchestra.run.smithi006.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-07T23:46:18.163 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_CODENAME=focal 2022-02-07T23:46:18.163 INFO:teuthology.orchestra.run.smithi006.stdout:UBUNTU_CODENAME=focal 2022-02-07T23:46:18.164 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2022-02-07T23:46:18.183 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: set os type and version on server 2022-02-07T23:46:18.183 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2022-02-07T23:46:18.359 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2022-02-07T23:46:18.365 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="Ubuntu" 2022-02-07T23:46:18.365 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-07T23:46:18.366 INFO:teuthology.orchestra.run.smithi018.stdout:ID=ubuntu 2022-02-07T23:46:18.366 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE=debian 2022-02-07T23:46:18.366 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-07T23:46:18.366 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="20.04" 2022-02-07T23:46:18.367 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-07T23:46:18.367 INFO:teuthology.orchestra.run.smithi018.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-07T23:46:18.367 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-07T23:46:18.367 INFO:teuthology.orchestra.run.smithi018.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-07T23:46:18.368 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_CODENAME=focal 2022-02-07T23:46:18.368 INFO:teuthology.orchestra.run.smithi018.stdout:UBUNTU_CODENAME=focal 2022-02-07T23:46:18.368 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2022-02-07T23:46:18.468 INFO:teuthology.dispatcher.supervisor:Running job 6666771 2022-02-07T23:46:18.468 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-07_19:43:03-rados-WIP_GBH_safe_shutdown_v2B-distro-basic-smithi/6666771 --name benhanokh-2022-02-07_19:43:03-rados-WIP_GBH_safe_shutdown_v2B-distro-basic-smithi --description rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} -- /home/teuthworker/archive/benhanokh-2022-02-07_19:43:03-rados-WIP_GBH_safe_shutdown_v2B-distro-basic-smithi/6666771/orig.config.yaml 2022-02-07T23:46:18.471 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-07_19:43:03-rados-WIP_GBH_safe_shutdown_v2B-distro-basic-smithi/6666771 2022-02-07T23:46:18.472 INFO:teuthology.dispatcher.supervisor:Job PID: 5809 2022-02-07T23:46:18.473 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-08T06:16:47.625 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-08T06:16:47.806 INFO:teuthology.kill:Killing Pids: {5809} 2022-02-08T06:16:48.943 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2022-02-08T06:16:48.944 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['osd.3', 'osd.4', 'osd.5', 'client.0'] 2022-02-08T06:16:48.944 INFO:teuthology.misc:Compressing logs... 2022-02-08T06:16:48.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2022-02-08T06:16:49.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2022-02-08T06:16:49.406 INFO:teuthology.misc:Compressing logs... 2022-02-08T06:16:50.306 INFO:teuthology.misc:Compressing logs... 2022-02-08T06:21:25.445 INFO:teuthology.kill:No teuthology processes running 2022-02-08T06:21:26.731 INFO:teuthology.kill:Nuking machines: ['smithi006', 'smithi018'] 2022-02-08T06:21:27.756 INFO:teuthology.kill:2022-02-08 06:21:27,756.756 INFO:teuthology.nuke:targets: 2022-02-08T06:21:27.757 INFO:teuthology.kill: smithi006.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKY81bkoTeePxm6G4AT+SWr4RSLg6VfglExyupfp+39hQM96Ar/IEo3GyXiJy5ZP+GhGSLRhnLlKH4go9SPFEwZH520+MPSlgq7GoJTyJkQ9ie6j/8F9yET7aXK4BomEnp5IPkCWwcy6MnuvIfANkPn9nYl8GeZjgicyqC1bxN838BktJZu8sMXsETmqTdpUOxAv4KiZ+voU0RWSnao2caIQyvY/Lk0jtZMX2v3av7cAZwkRm6AaghHw74QstxbTdLvE/e3g6svxxNRhkV9btGkwkLUNtPoKYB3s88iFYjueHZ2b6O950tKUjVnUF5ErKIROJuDAVzrQyJZuj223DV4BXi1P1mGGMMCD5VFzBiEeXZXFHd/59fMI97oi73Bwv/1X27UaMPDrcCh2hj6C4wlRInxDbfGyBycpBSUgC8hGXyN+DBsbMiAKWoWsWYUoy8qr3ES5YGhjCAikaROTJlWYtSWnpEx4J9Jakp3zVfTC52q43pYs241rpcAkECO30= 2022-02-08T06:21:27.757 INFO:teuthology.kill: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7R4cKDMSuyiXtSlTSItzmfMxJ07cW7Ghl/YIpKYGtKFgz2/QZ+IhnCtPmDLPkHgC5mvpW5qL26UzKAfdHakVgGd5A/xua1KiQwBydmaKGtpCRkFKvHv2cM2TIjBftb0uZlnB7pyTTzH5+S/e/kGLbTuBp2clgEyxdZWdfeQDNU/mTAr8PnEVZUZWEDeWHWMV1uP/gNTDXzH9AOLzS3Osy7OXe5Ji1DtvMkPwiwZSiqrT3r66VjOo88JwCR4u1RDq0yWg0NJCadW+4I0qO8MInsOyeWRIa1nHRl+V29dz6KcyLZYucy21ah3Jr/pR67KfkfSPmIhm52QZX79Q6PQf5xtW2rPVXpm7GTHb+KMNwakbmvoZCQHWCVt8CEdWckH7Wu9FadrYLzZ6SoaB5PbivYQwvgpzyWnssvuZsjOjD8ZpYEZRFIq3hlFkqbYzI5WKW1q+u2frTJ9srjVdXNwYtpXm7gpvZyTHpAbwJ36nUlkhC51mP6OTntf9Ce3W34vs= 2022-02-08T06:21:27.785 INFO:teuthology.kill:2022-02-08 06:21:27,785.785 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-08T06:21:27.791 INFO:teuthology.kill:2022-02-08 06:21:27,791.791 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-08T06:21:27.872 INFO:teuthology.kill:2022-02-08 06:21:27,872.872 INFO:teuthology.orchestra.console:Power off smithi006 2022-02-08T06:21:27.891 INFO:teuthology.kill:2022-02-08 06:21:27,891.891 INFO:teuthology.orchestra.console:Power off smithi018 2022-02-08T06:21:36.267 INFO:teuthology.kill:2022-02-08 06:21:36,266.266 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2022-02-08T06:21:36.413 INFO:teuthology.kill:2022-02-08 06:21:36,412.412 INFO:teuthology.lock.ops:unlocked: smithi018.front.sepia.ceph.com 2022-02-08T06:21:40.366 INFO:teuthology.kill:2022-02-08 06:21:40,365.365 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2022-02-08T06:21:40.512 INFO:teuthology.kill:2022-02-08 06:21:40,512.512 INFO:teuthology.lock.ops:unlocked: smithi006.front.sepia.ceph.com 2022-02-08T06:23:40.853 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-08T06:23:40.959 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi006 but it was locked by another job: /home/teuthworker/archive/ozeneva-2022-02-07_23:28:44-rgw-wip-omri-tracing-compiled-distro-basic-smithi/6668011 2022-02-08T06:23:40.960 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi018 but it was locked by another job: /home/teuthworker/archive/ozeneva-2022-02-07_23:28:44-rgw-wip-omri-tracing-compiled-distro-basic-smithi/6668011