2021-12-05T18:38:13.839 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-05T18:38:13.963 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi071.front.sepia.ceph.com' reimaging 2021-12-05T18:38:13.964 INFO:teuthology.lock.ops:Updating [ubuntu@smithi071.front.sepia.ceph.com]: reset os type and version on server 2021-12-05T18:38:13.964 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2021-12-05T18:38:13.983 INFO:teuthology.lock.ops:Node 'ubuntu@smithi071.front.sepia.ceph.com' reimaging is complete 2021-12-05T18:38:13.983 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi093.front.sepia.ceph.com' reimaging 2021-12-05T18:38:13.983 INFO:teuthology.lock.ops:Updating [ubuntu@smithi093.front.sepia.ceph.com]: reset os type and version on server 2021-12-05T18:38:13.984 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2021-12-05T18:38:14.002 INFO:teuthology.lock.ops:Node 'ubuntu@smithi093.front.sepia.ceph.com' reimaging is complete 2021-12-05T18:38:14.190 INFO:teuthology.provision.fog.smithi071:Scheduling deploy of ubuntu 20.04 2021-12-05T18:38:14.211 INFO:teuthology.provision.fog.smithi093:Scheduling deploy of ubuntu 20.04 2021-12-05T18:38:14.528 INFO:teuthology.orchestra.console:Power off smithi093 2021-12-05T18:38:14.544 INFO:teuthology.orchestra.console:Power off smithi071 2021-12-05T18:38:14.674 INFO:teuthology.orchestra.console:Power off for smithi093 completed 2021-12-05T18:38:14.686 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2021-12-05T18:38:14.776 INFO:teuthology.orchestra.console:Power on smithi093 2021-12-05T18:38:14.787 INFO:teuthology.orchestra.console:Power on smithi071 2021-12-05T18:38:23.148 INFO:teuthology.orchestra.console:Power on for smithi093 completed 2021-12-05T18:38:23.156 INFO:teuthology.orchestra.console:Power on for smithi071 completed 2021-12-05T18:38:23.250 INFO:teuthology.provision.fog.smithi093:Waiting for deploy to finish 2021-12-05T18:38:23.257 INFO:teuthology.provision.fog.smithi071:Waiting for deploy to finish 2021-12-05T18:43:42.833 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-05T18:43:42.914 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-05T18:43:43.333 INFO:teuthology.provision.fog.smithi071:Node is ready 2021-12-05T18:43:43.339 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071.front.sepia.ceph.com 2021-12-05T18:43:43.346 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71 smithi071.front.sepia.ceph.com smithi071 2021-12-05T18:43:43.492 INFO:teuthology.provision.fog.smithi093:Node is ready 2021-12-05T18:43:43.499 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093.front.sepia.ceph.com 2021-12-05T18:43:43.505 INFO:teuthology.provision.fog.smithi071:Deploy complete! 2021-12-05T18:43:43.507 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93 smithi093.front.sepia.ceph.com smithi093 2021-12-05T18:43:43.739 INFO:teuthology.provision.fog.smithi093:Deploy complete! 2021-12-05T18:43:43.923 INFO:teuthology.lock.ops:Checking smithi071.front.sepia.ceph.com 2021-12-05T18:43:43.924 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-05T18:43:43.947 INFO:teuthology.lock.ops:Checking smithi093.front.sepia.ceph.com 2021-12-05T18:43:43.948 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-05T18:43:43.966 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: set os type and version on server 2021-12-05T18:43:43.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2021-12-05T18:43:44.155 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2021-12-05T18:43:44.162 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Ubuntu" 2021-12-05T18:43:44.163 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-12-05T18:43:44.163 INFO:teuthology.orchestra.run.smithi071.stdout:ID=ubuntu 2021-12-05T18:43:44.164 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE=debian 2021-12-05T18:43:44.164 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-12-05T18:43:44.164 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="20.04" 2021-12-05T18:43:44.165 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.ubuntu.com/" 2021-12-05T18:43:44.165 INFO:teuthology.orchestra.run.smithi071.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-12-05T18:43:44.166 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-12-05T18:43:44.166 INFO:teuthology.orchestra.run.smithi071.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-12-05T18:43:44.167 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_CODENAME=focal 2021-12-05T18:43:44.167 INFO:teuthology.orchestra.run.smithi071.stdout:UBUNTU_CODENAME=focal 2021-12-05T18:43:44.168 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2021-12-05T18:43:44.187 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: set os type and version on server 2021-12-05T18:43:44.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2021-12-05T18:43:44.353 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2021-12-05T18:43:44.360 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Ubuntu" 2021-12-05T18:43:44.360 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-12-05T18:43:44.360 INFO:teuthology.orchestra.run.smithi093.stdout:ID=ubuntu 2021-12-05T18:43:44.360 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE=debian 2021-12-05T18:43:44.361 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-12-05T18:43:44.361 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="20.04" 2021-12-05T18:43:44.361 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.ubuntu.com/" 2021-12-05T18:43:44.361 INFO:teuthology.orchestra.run.smithi093.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-12-05T18:43:44.362 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-12-05T18:43:44.362 INFO:teuthology.orchestra.run.smithi093.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-12-05T18:43:44.362 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_CODENAME=focal 2021-12-05T18:43:44.362 INFO:teuthology.orchestra.run.smithi093.stdout:UBUNTU_CODENAME=focal 2021-12-05T18:43:44.363 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2021-12-05T18:43:44.460 INFO:teuthology.dispatcher.supervisor:Running job 6546774 2021-12-05T18:43:44.461 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546774 --name benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} -- /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546774/orig.config.yaml 2021-12-05T18:43:44.464 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546774 2021-12-05T18:43:44.465 INFO:teuthology.dispatcher.supervisor:Job PID: 4437 2021-12-05T18:43:44.465 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-06T06:44:38.348 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-06T06:44:38.578 INFO:teuthology.kill:Killing Pids: {4437} 2021-12-06T06:44:39.825 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-12-06T06:44:39.826 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-12-06T06:44:39.826 INFO:teuthology.misc:Compressing logs... 2021-12-06T06:44:39.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2021-12-06T06:44:40.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2021-12-06T06:44:40.275 INFO:teuthology.misc:Compressing logs... 2021-12-06T06:44:41.777 INFO:teuthology.misc:Compressing logs... 2021-12-06T06:44:52.494 INFO:teuthology.kill:No teuthology processes running 2021-12-06T06:44:53.684 INFO:teuthology.kill:Nuking machines: ['smithi071', 'smithi093'] 2021-12-06T06:44:54.681 INFO:teuthology.kill:2021-12-06 06:44:54,681.681 INFO:teuthology.nuke:targets: 2021-12-06T06:44:54.682 INFO:teuthology.kill: smithi071.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC64q5Hi/YkT2bThaw1aJz4pTcyyJvGQfcihpkSUhXCiNgoX0W0tOsJretTBKKjgQWn1iFOjyLtFeLRCKnm6/JBDUTDt37wJfV3OdJFrab/0DFdthz1E9EZHrX/u/d9iPiMXgkD5dqCtMy5McPaT2DKmjU0AA5GsUegOvMGVzUCBU5Cah6LBRicmje3kUVU4nf/2Ykj9xeCZUeVWUGop4QWQFjpSkdrIvWwi32XdSaTpOHBQXgXfMmkdO+QCmuyzzON+l+0jOyUzSUc3grj0zYiNhikRbOfAoSKNf+B8II/pUlusuoq80d29FSIIOxS9KUy1CWnFKyIjLjopkM/WzBHFcT01S6Nh1kdEBjMExPyndyeEJoPU/ylt0iSk6unngW4n+bxrPgKJQuD5PP5AjbGbA0yqVqDgvxzLZXJSyR5IMGD9MLdID5TdrSneZt6lA9QUm5Yy3o4cuXAuIjHlgg51ellaIYArEHNwnLckJJsXNBsbVdDTUu3VKbPrxBc+O8= 2021-12-06T06:44:54.682 INFO:teuthology.kill: smithi093.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC++ya6e1qV4Muj4gyTQJjGPejWzLfNz5uRtiQUq+mrTRR7dO3ePp/Omajw8ZF/XfF3adGbIBZcP+nNlu6Z8K5JIV8l5ak1hhbokQlHYK1chMKbJwVXLQfL/JozTKSskvv6E6l5vc/UPQvqKoaZH/vvTy9KeLGNbdzlLHhtkiDP2wcralOizWQjNphfMbixSWIg5Tr1cF9kE9bQWVGIm3YrrHkN7EGnXY4Zpwof4eRh9ETXONpOtj7d3uawrDJyxcpBrd7qIrZ5thq/fzBmuhKj7AQRDWU2ALcwNPVjZjrjWxobNshCcp1+XyYYxwVrZ/89pq70u3Jrb3Lv/g8b1hEFea6McZlU1PnNJQa2/KeRe1qOaUtx/RSoC10zVS2zczDnStsmGWqHtIqSatzfQi/pnag0AvwQnnHzrEH5a33oZR1Jrhd/OOFQFn5K0ZghHF1MDGYRrUgMwpWWigvOs74FSLCX5AMW9mAy2kZie+PyIeXJU28keNHVsSq9OZRZa5U= 2021-12-06T06:44:54.708 INFO:teuthology.kill:2021-12-06 06:44:54,707.707 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-06T06:44:54.711 INFO:teuthology.kill:2021-12-06 06:44:54,710.710 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-06T06:44:54.789 INFO:teuthology.kill:2021-12-06 06:44:54,788.788 INFO:teuthology.orchestra.console:Power off smithi093 2021-12-06T06:44:54.803 INFO:teuthology.kill:2021-12-06 06:44:54,803.803 INFO:teuthology.orchestra.console:Power off smithi071 2021-12-06T06:45:03.161 INFO:teuthology.kill:2021-12-06 06:45:03,161.161 INFO:teuthology.orchestra.console:Power off for smithi093 completed 2021-12-06T06:45:03.166 INFO:teuthology.kill:2021-12-06 06:45:03,166.166 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2021-12-06T06:45:03.321 INFO:teuthology.kill:2021-12-06 06:45:03,320.320 INFO:teuthology.lock.ops:unlocked: smithi071.front.sepia.ceph.com 2021-12-06T06:45:03.324 INFO:teuthology.kill:2021-12-06 06:45:03,324.324 INFO:teuthology.lock.ops:unlocked: smithi093.front.sepia.ceph.com 2021-12-06T06:47:03.664 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-06T06:47:03.774 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi093 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_06:07:02-rbd-pacific-distro-default-smithi/6547436