2022-02-04T17:43:25.689 INFO:root:teuthology version: 1.1.0-3094160c
2022-02-04T17:43:25.809 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging
2022-02-04T17:43:25.811 INFO:teuthology.lock.ops:Updating [ubuntu@smithi084.front.sepia.ceph.com]: reset os type and version on server
2022-02-04T17:43:25.811 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2022-02-04T17:43:25.830 INFO:teuthology.lock.ops:Node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging is complete
2022-02-04T17:43:25.831 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi089.front.sepia.ceph.com' reimaging
2022-02-04T17:43:25.831 INFO:teuthology.lock.ops:Updating [ubuntu@smithi089.front.sepia.ceph.com]: reset os type and version on server
2022-02-04T17:43:25.832 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2022-02-04T17:43:25.853 INFO:teuthology.lock.ops:Node 'ubuntu@smithi089.front.sepia.ceph.com' reimaging is complete
2022-02-04T17:43:26.036 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of ubuntu 20.04
2022-02-04T17:43:26.074 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of ubuntu 20.04
2022-02-04T17:43:26.409 INFO:teuthology.orchestra.console:Power off smithi084
2022-02-04T17:43:26.425 INFO:teuthology.orchestra.console:Power off smithi089
2022-02-04T17:43:34.818 INFO:teuthology.orchestra.console:Power off for smithi089 completed
2022-02-04T17:43:34.920 INFO:teuthology.orchestra.console:Power on smithi089
2022-02-04T17:43:38.896 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2022-02-04T17:43:38.997 INFO:teuthology.orchestra.console:Power on smithi084
2022-02-04T17:43:43.253 INFO:teuthology.orchestra.console:Power on for smithi084 completed
2022-02-04T17:43:43.352 INFO:teuthology.orchestra.console:Power on for smithi089 completed
2022-02-04T17:43:43.355 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish
2022-02-04T17:43:43.454 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish
2022-02-04T17:48:34.836 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-04T17:48:35.330 INFO:teuthology.provision.fog.smithi089:Node is ready
2022-02-04T17:48:35.336 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com
2022-02-04T17:48:35.342 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089
2022-02-04T17:48:35.508 INFO:teuthology.provision.fog.smithi089:Deploy complete!
2022-02-04T17:48:47.061 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-04T17:48:47.571 INFO:teuthology.provision.fog.smithi084:Node is ready
2022-02-04T17:48:47.577 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com
2022-02-04T17:48:47.586 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084
2022-02-04T17:48:47.757 INFO:teuthology.provision.fog.smithi084:Deploy complete!
2022-02-04T17:48:47.948 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com
2022-02-04T17:48:47.949 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-04T17:48:47.969 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com
2022-02-04T17:48:47.970 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-04T17:48:47.988 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server
2022-02-04T17:48:47.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com'
2022-02-04T17:48:48.166 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64
2022-02-04T17:48:48.173 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Ubuntu"
2022-02-04T17:48:48.173 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-04T17:48:48.174 INFO:teuthology.orchestra.run.smithi084.stdout:ID=ubuntu
2022-02-04T17:48:48.174 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE=debian
2022-02-04T17:48:48.174 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-04T17:48:48.175 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="20.04"
2022-02-04T17:48:48.175 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-04T17:48:48.175 INFO:teuthology.orchestra.run.smithi084.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-04T17:48:48.175 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-04T17:48:48.175 INFO:teuthology.orchestra.run.smithi084.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-04T17:48:48.176 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_CODENAME=focal
2022-02-04T17:48:48.176 INFO:teuthology.orchestra.run.smithi084.stdout:UBUNTU_CODENAME=focal
2022-02-04T17:48:48.177 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2022-02-04T17:48:48.194 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server
2022-02-04T17:48:48.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com'
2022-02-04T17:48:48.370 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64
2022-02-04T17:48:48.377 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Ubuntu"
2022-02-04T17:48:48.377 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-04T17:48:48.378 INFO:teuthology.orchestra.run.smithi089.stdout:ID=ubuntu
2022-02-04T17:48:48.378 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE=debian
2022-02-04T17:48:48.378 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-04T17:48:48.378 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="20.04"
2022-02-04T17:48:48.379 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-04T17:48:48.379 INFO:teuthology.orchestra.run.smithi089.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-04T17:48:48.379 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-04T17:48:48.379 INFO:teuthology.orchestra.run.smithi089.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-04T17:48:48.380 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_CODENAME=focal
2022-02-04T17:48:48.380 INFO:teuthology.orchestra.run.smithi089.stdout:UBUNTU_CODENAME=focal
2022-02-04T17:48:48.380 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2022-02-04T17:48:48.467 INFO:teuthology.dispatcher.supervisor:Running job 6661652
2022-02-04T17:48:48.468 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2022-02-04_17:39:45-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6661652 --name nojha-2022-02-04_17:39:45-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi --description orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} -- /home/teuthworker/archive/nojha-2022-02-04_17:39:45-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6661652/orig.config.yaml
2022-02-04T17:48:48.473 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2022-02-04_17:39:45-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6661652
2022-02-04T17:48:48.473 INFO:teuthology.dispatcher.supervisor:Job PID: 1282
2022-02-04T17:48:48.474 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-05T00:18:59.695 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-05T00:19:09.697 INFO:teuthology.kill:Killing Pids: {1282}
2022-02-05T00:19:10.867 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2022-02-05T00:19:10.868 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2022-02-05T00:19:10.869 INFO:teuthology.misc:Compressing logs...
2022-02-05T00:19:10.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com'
2022-02-05T00:19:11.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com'
2022-02-05T00:19:11.061 INFO:teuthology.orchestra.run.smithi084.stderr:find: '/var/lib/ceph/crash': No such file or directory
2022-02-05T00:19:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:find: '/var/lib/ceph/crash': No such file or directory
2022-02-05T00:19:11.354 INFO:teuthology.orchestra.run.smithi084.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-05T00:19:11.355 INFO:teuthology.orchestra.run.smithi084.stderr:tar: Error is not recoverable: exiting now
2022-02-05T00:19:11.374 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-05T00:19:11.375 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now
2022-02-05T00:19:11.377 INFO:teuthology.misc:Compressing logs...
2022-02-05T00:19:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-05T00:19:11.605 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-05T00:19:11.760 INFO:teuthology.misc:Compressing logs...
2022-02-05T00:19:14.262 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2022-02-05T00:19:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-05T00:19:29.969 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-05T00:19:34.223 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2022-02-05T00:19:39.343 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-05T00:19:43.265 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2022-02-05T00:19:46.899 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2022-02-05T00:19:53.151 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-05T00:20:00.950 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-05T00:20:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-05T00:20:14.698 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-05T00:20:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/238cc18c-85e3-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-05T00:20:56.749 INFO:teuthology.kill:No teuthology processes running
2022-02-05T00:20:57.954 INFO:teuthology.kill:Nuking machines: ['smithi084', 'smithi089']
2022-02-05T00:20:58.948 INFO:teuthology.kill:2022-02-05 00:20:58,948.948 INFO:teuthology.nuke:targets:
2022-02-05T00:20:58.949 INFO:teuthology.kill:  smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRtBcNobluUkGzg+affG1C5vpTFiWcI3LiuVuGcWU8deWe0ZzJDRVhtVRoDNQIdI3ZBbse/2rwFIp+3q60UAFA6h+4gcM/Ujk575QNXDU9m5pCfGWYejR2fNNrH8adgYsTB/MiLwdfqUtO3IfZZ3KIl/2ACwBwVMSpSwniyL4BD75nDt1GPx+ynCd5a37Q5yFzsdXa0hOITP3Hcg6dL3KFCFH+3hPTFNZh3b360yLfjVKDNFRq8J5FU6adnXhvA7quZ+Y+xWhvWbrUqspiLOd4Gb3TgQSrZAY6iAN2AbTzq5s4NzLBoVtpT+eS1rlWdSR0Qn4qpY6UOYBmZqzEslLCMNhVVzl3pJxj7PB0+Am+6MgOD5pLtYgRRS9Tnm/BpkBh08YuccDegDi718E9+ItYJROUfy31dOOsyafqY492DHr+CkIFyBeXwKbkSYW5jaUtdP2bGqd4SyXgnI9k8uLyiLFt875tbdRXw40yA9pZXBRzeCdlz2IAnWEHGbVcSgk=
2022-02-05T00:20:58.950 INFO:teuthology.kill:  smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1aGuEHKn3MIVJKvTmFtztZIdTz4C+csxjZr4mPwZM1lJ8NVeTz5Xvd8v6OYVk45SutGWRFEOXfLjYEE1oMlY151ZzcgQggB6IAiFVplkWsvz+9TpFdpZADZvCSCx+7N5R5n+usAXecdsIfX/EwyXoHGAPldrT+nGt0TvwuF2ij0hgpgoKmjjpjxtfDXIjnWUZc8Y78CBuoMLWFzd0jO7IyuycGDdqpMjHZp1/5r9A00krnTZ/9LCJYfEJeQ6k1rydlpdkFmIIXBCP9K1oI1ScVZIRTs//XtnnSllTKbN2UpYWgSwP/E0XZIT5XcukmAhNIIY8j+2RhLLEO94jSYkYgXDjziQcSdgpItzi7WQFO4eugyfVhrixKg6SksGL5aoVth8qDnjfBhGqB3AAXO2HUwEROqcJk5wk8jHcMPWFQj3mjMvZWYwFo/X7HMurH5r0lv712BanY4yxisntIzSzvxUHaNHFmKiPScnKprnOa4bJ6nu9xvW7yDWwO6AFUK8=
2022-02-05T00:20:58.975 INFO:teuthology.kill:2022-02-05 00:20:58,975.975 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-05T00:20:58.981 INFO:teuthology.kill:2022-02-05 00:20:58,981.981 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-05T00:20:59.053 INFO:teuthology.kill:2022-02-05 00:20:59,052.052 INFO:teuthology.orchestra.console:Power off smithi084
2022-02-05T00:20:59.068 INFO:teuthology.kill:2022-02-05 00:20:59,067.067 INFO:teuthology.orchestra.console:Power off smithi089
2022-02-05T00:21:07.434 INFO:teuthology.kill:2022-02-05 00:21:07,434.434 INFO:teuthology.orchestra.console:Power off for smithi089 completed
2022-02-05T00:21:07.580 INFO:teuthology.kill:2022-02-05 00:21:07,579.579 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com
2022-02-05T00:21:11.535 INFO:teuthology.kill:2022-02-05 00:21:11,535.535 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2022-02-05T00:21:11.702 INFO:teuthology.kill:2022-02-05 00:21:11,701.701 INFO:teuthology.lock.ops:unlocked: smithi084.front.sepia.ceph.com
2022-02-05T00:23:12.024 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-05T00:23:12.152 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi084 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-04_22:20:21-rados-wip-yuri8-testing-2022-02-04-1220-quincy-distro-default-smithi/6661978
2022-02-05T00:23:12.153 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi089 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-04_22:20:21-rados-wip-yuri8-testing-2022-02-04-1220-quincy-distro-default-smithi/6661978