2021-11-02T10:21:07.925 INFO:root:teuthology version: 1.1.0-c56135d1
2021-11-02T10:21:08.048 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging
2021-11-02T10:21:08.050 INFO:teuthology.lock.ops:Updating [ubuntu@smithi146.front.sepia.ceph.com]: reset os type and version on server
2021-11-02T10:21:08.051 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2021-11-02T10:21:08.073 INFO:teuthology.lock.ops:Node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging is complete
2021-11-02T10:21:08.073 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi160.front.sepia.ceph.com' reimaging
2021-11-02T10:21:08.074 INFO:teuthology.lock.ops:Updating [ubuntu@smithi160.front.sepia.ceph.com]: reset os type and version on server
2021-11-02T10:21:08.074 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server
2021-11-02T10:21:08.095 INFO:teuthology.lock.ops:Node 'ubuntu@smithi160.front.sepia.ceph.com' reimaging is complete
2021-11-02T10:21:08.290 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of ubuntu 20.04
2021-11-02T10:21:08.295 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of ubuntu 20.04
2021-11-02T10:21:08.647 INFO:teuthology.orchestra.console:Power off smithi146
2021-11-02T10:21:08.660 INFO:teuthology.orchestra.console:Power off smithi160
2021-11-02T10:21:08.790 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2021-11-02T10:21:08.803 INFO:teuthology.orchestra.console:Power off for smithi160 completed
2021-11-02T10:21:08.891 INFO:teuthology.orchestra.console:Power on smithi146
2021-11-02T10:21:08.904 INFO:teuthology.orchestra.console:Power on smithi160
2021-11-02T10:21:17.305 INFO:teuthology.orchestra.console:Power on for smithi160 completed
2021-11-02T10:21:17.362 INFO:teuthology.orchestra.console:Power on for smithi146 completed
2021-11-02T10:21:17.407 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish
2021-11-02T10:21:17.463 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2021-11-02T10:26:52.730 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-02T10:26:53.249 INFO:teuthology.provision.fog.smithi160:Node is ready
2021-11-02T10:26:53.255 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com
2021-11-02T10:26:53.262 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160
2021-11-02T10:26:53.514 INFO:teuthology.provision.fog.smithi160:Deploy complete!
2021-11-02T10:27:21.076 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-02T10:27:21.720 INFO:teuthology.provision.fog.smithi146:Node is ready
2021-11-02T10:27:21.727 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2021-11-02T10:27:21.734 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2021-11-02T10:27:21.938 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2021-11-02T10:27:22.126 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2021-11-02T10:27:22.127 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-02T10:27:22.145 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com
2021-11-02T10:27:22.145 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-02T10:27:22.163 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2021-11-02T10:27:22.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2021-11-02T10:27:22.338 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2021-11-02T10:27:22.345 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Ubuntu"
2021-11-02T10:27:22.345 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-11-02T10:27:22.346 INFO:teuthology.orchestra.run.smithi146.stdout:ID=ubuntu
2021-11-02T10:27:22.346 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE=debian
2021-11-02T10:27:22.346 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-11-02T10:27:22.346 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="20.04"
2021-11-02T10:27:22.347 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.ubuntu.com/"
2021-11-02T10:27:22.347 INFO:teuthology.orchestra.run.smithi146.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-11-02T10:27:22.347 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-11-02T10:27:22.347 INFO:teuthology.orchestra.run.smithi146.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-11-02T10:27:22.348 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_CODENAME=focal
2021-11-02T10:27:22.348 INFO:teuthology.orchestra.run.smithi146.stdout:UBUNTU_CODENAME=focal
2021-11-02T10:27:22.349 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2021-11-02T10:27:22.366 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server
2021-11-02T10:27:22.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com'
2021-11-02T10:27:22.630 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64
2021-11-02T10:27:22.635 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Ubuntu"
2021-11-02T10:27:22.636 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-11-02T10:27:22.636 INFO:teuthology.orchestra.run.smithi160.stdout:ID=ubuntu
2021-11-02T10:27:22.636 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE=debian
2021-11-02T10:27:22.637 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-11-02T10:27:22.637 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="20.04"
2021-11-02T10:27:22.637 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.ubuntu.com/"
2021-11-02T10:27:22.637 INFO:teuthology.orchestra.run.smithi160.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-11-02T10:27:22.638 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-11-02T10:27:22.638 INFO:teuthology.orchestra.run.smithi160.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-11-02T10:27:22.638 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_CODENAME=focal
2021-11-02T10:27:22.638 INFO:teuthology.orchestra.run.smithi160.stdout:UBUNTU_CODENAME=focal
2021-11-02T10:27:22.639 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server
2021-11-02T10:27:22.726 INFO:teuthology.dispatcher.supervisor:Running job 6474849
2021-11-02T10:27:22.727 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_pdiazbou@teuthology --archive /home/teuthworker/archive/pdiazbou-2021-11-02_04:20:02-orch-wip-pereman2-testing-2021-11-01-2310-distro-basic-smithi/6474849 --name pdiazbou-2021-11-02_04:20:02-orch-wip-pereman2-testing-2021-11-01-2310-distro-basic-smithi --description orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} -- /home/teuthworker/archive/pdiazbou-2021-11-02_04:20:02-orch-wip-pereman2-testing-2021-11-01-2310-distro-basic-smithi/6474849/orig.config.yaml
2021-11-02T10:27:22.731 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdiazbou-2021-11-02_04:20:02-orch-wip-pereman2-testing-2021-11-01-2310-distro-basic-smithi/6474849
2021-11-02T10:27:22.731 INFO:teuthology.dispatcher.supervisor:Job PID: 3065
2021-11-02T10:27:22.732 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-02T22:28:16.715 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-02T22:28:16.898 INFO:teuthology.kill:Killing Pids: {3065}
2021-11-02T22:28:18.064 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2021-11-02T22:28:18.065 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2021-11-02T22:28:18.066 INFO:teuthology.misc:Compressing logs...
2021-11-02T22:28:18.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2021-11-02T22:28:18.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com'
2021-11-02T22:28:18.257 INFO:teuthology.orchestra.run.smithi146.stderr:find: '/var/lib/ceph/crash': No such file or directory
2021-11-02T22:28:18.429 INFO:teuthology.orchestra.run.smithi160.stderr:find: '/var/lib/ceph/crash': No such file or directory
2021-11-02T22:28:18.452 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-11-02T22:28:18.452 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now
2021-11-02T22:28:18.473 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-11-02T22:28:18.473 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now
2021-11-02T22:28:18.475 INFO:teuthology.misc:Compressing logs...
2021-11-02T22:28:19.152 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-11-02T22:28:19.169 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-11-02T22:28:19.627 INFO:teuthology.misc:Compressing logs...
2021-11-02T22:28:43.381 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-11-02T22:28:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-11-02T22:28:52.887 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2021-11-02T22:29:10.084 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-11-02T22:29:24.170 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-11-02T22:29:35.424 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-11-02T22:29:42.702 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2021-11-02T22:29:46.813 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2021-11-02T22:30:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-11-02T22:30:14.298 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/97d9658c-3bc8-11ec-8c28-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2021-11-02T22:31:10.515 INFO:teuthology.kill:No teuthology processes running
2021-11-02T22:31:11.728 INFO:teuthology.kill:Nuking machines: ['smithi146', 'smithi160']
2021-11-02T22:31:12.745 INFO:teuthology.kill:2021-11-02 22:31:12,745.745 INFO:teuthology.nuke:targets:
2021-11-02T22:31:12.746 INFO:teuthology.kill:  smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnPPo2XruWKWKlXdFPREpJjNk/5n+/6UjGFaKFR0EPDaH8W7FfzNwGWtNmmBFskw7nV0nF9EoD4/3SgUliXCoMj2NAO3FVd0HqUIZ9UUSUkZXugHTVQ0jce+D2KT0pAA3pODJ8n/XmYuzTzoZMka10NGykyeEzrzN8ZrlIxGsk59/14PeI//u4cmcfqnqD+CrIBHfT0FO7z//uRDeXmCTkpN+Ft21WDPX566f0xLfbgiWZV1/huDnAeVRohF8Q+ewJzmNHTp0PjIT3/EPWeBHRB9xE1ql//M9EJHFRbw0H7fNqSu1reSuTDBEsx54rL/+5vAQ5db5sSqjERH4HJiXom+Lcv6KhDy+y6ebh3rirq/iIHYS7lKRAte+aN0B5Qb/n/4ZPlEE/acdSPgnx2c7BaBjwrNa56VY7YWiurrHQI+cx52Ob4fC147xnKikuT/+kRqTSfJjFoDmuAhxx33f5129OcmLoPhyfHaRSxcf+HbACLwQE3PjBm6JjDO/TrVE=
2021-11-02T22:31:12.746 INFO:teuthology.kill:  smithi160.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzsQLPbnSif4eg5ldH3iLiUF3Yh+Ekj/GFod9sPuQ49B2SEUH5Po843sNfMT9nn9jnxhSBop7oJn+68nNwRhjSuxtYIW2gF7HByMkTn/mfjbHiJsrd2taLKkYJKEuX897pFLr5q+ibO3nBVeRFsZzBPEBlZZDK3+vE5Mkedwy2yiYGPf6HQb0a9CvpPlOreW2VW/ANtFiXLsImdIzzd8t8nNR++H4pGEtAHQFmRkR5yZrwkMX9eGAtzcPMqtlKUNagwzF9p/tqXY40jLNtSvPHeE2pnYwzqsyxlU3t2SiKlkpWHd0Yowu0CfZ4eSVk2uZy0mD66uVR6Is9u0dWgqks9yDlmhqsdu4gasRV/Gujmcoh+dsMwNBybkiojNXQH73wa8AeZ0l0MpsKiqIY7LAWzAhIcQE3iDafDilLLYNdLYsLfB8QCtChagHIKXqBpIEPXkU0Ydef3fNwHp2do4qjrnENdX5SjWAFdQ2pkoYuQHtobiXGFCekbSws6KbVi90=
2021-11-02T22:31:12.774 INFO:teuthology.kill:2021-11-02 22:31:12,774.774 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-02T22:31:12.778 INFO:teuthology.kill:2021-11-02 22:31:12,778.778 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-02T22:31:12.851 INFO:teuthology.kill:2021-11-02 22:31:12,851.851 INFO:teuthology.orchestra.console:Power off smithi146
2021-11-02T22:31:12.865 INFO:teuthology.kill:2021-11-02 22:31:12,865.865 INFO:teuthology.orchestra.console:Power off smithi160
2021-11-02T22:31:25.333 INFO:teuthology.kill:2021-11-02 22:31:25,333.333 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2021-11-02T22:31:25.339 INFO:teuthology.kill:2021-11-02 22:31:25,339.339 INFO:teuthology.orchestra.console:Power off for smithi160 completed
2021-11-02T22:31:25.494 INFO:teuthology.kill:2021-11-02 22:31:25,493.493 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com
2021-11-02T22:31:25.495 INFO:teuthology.kill:2021-11-02 22:31:25,495.495 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com
2021-11-02T22:33:25.809 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-02T22:33:25.901 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi146 but it was locked by another job: /home/teuthworker/archive/cbodley-2021-11-02_19:40:04-rgw-pr-37184-distro-basic-smithi/6478747
2021-11-02T22:33:25.902 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi160 but it was locked by another job: /home/teuthworker/archive/cbodley-2021-11-02_19:40:04-rgw-pr-37184-distro-basic-smithi/6478747