2022-02-10T06:02:17.509 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-10T06:02:17.634 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi117.front.sepia.ceph.com' reimaging 2022-02-10T06:02:17.635 INFO:teuthology.lock.ops:Updating [ubuntu@smithi117.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T06:02:17.635 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2022-02-10T06:02:17.655 INFO:teuthology.lock.ops:Node 'ubuntu@smithi117.front.sepia.ceph.com' reimaging is complete 2022-02-10T06:02:17.655 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging 2022-02-10T06:02:17.655 INFO:teuthology.lock.ops:Updating [ubuntu@smithi170.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T06:02:17.656 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2022-02-10T06:02:17.675 INFO:teuthology.lock.ops:Node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging is complete 2022-02-10T06:02:17.897 INFO:teuthology.provision.fog.smithi117:Scheduling deploy of ubuntu 18.04 2022-02-10T06:02:17.901 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of ubuntu 18.04 2022-02-10T06:02:18.270 INFO:teuthology.orchestra.console:Power off smithi117 2022-02-10T06:02:18.331 INFO:teuthology.orchestra.console:Power off smithi170 2022-02-10T06:02:26.701 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2022-02-10T06:02:26.802 INFO:teuthology.orchestra.console:Power on smithi170 2022-02-10T06:02:30.785 INFO:teuthology.orchestra.console:Power off for smithi117 completed 2022-02-10T06:02:30.887 INFO:teuthology.orchestra.console:Power on smithi117 2022-02-10T06:02:35.143 INFO:teuthology.orchestra.console:Power on for smithi117 completed 2022-02-10T06:02:35.163 INFO:teuthology.orchestra.console:Power on for smithi170 completed 2022-02-10T06:02:35.245 INFO:teuthology.provision.fog.smithi117:Waiting for deploy to finish 2022-02-10T06:02:35.264 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish 2022-02-10T06:06:59.657 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-10T06:07:05.673 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-10T06:07:08.872 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-10T06:07:11.688 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-10T06:07:14.884 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-10T06:07:20.724 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-10T06:07:20.897 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-10T06:07:26.905 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T06:07:27.156 INFO:teuthology.provision.fog.smithi170:Node is ready 2022-02-10T06:07:27.204 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com 2022-02-10T06:07:27.253 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170 2022-02-10T06:07:27.537 INFO:teuthology.provision.fog.smithi170:Deploy complete! 2022-02-10T06:07:32.918 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-10T06:07:39.057 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T06:07:39.311 INFO:teuthology.provision.fog.smithi117:Node is ready 2022-02-10T06:07:39.359 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117.front.sepia.ceph.com 2022-02-10T06:07:39.408 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117 smithi117.front.sepia.ceph.com smithi117 2022-02-10T06:07:39.685 INFO:teuthology.provision.fog.smithi117:Deploy complete! 2022-02-10T06:07:39.897 INFO:teuthology.lock.ops:Checking smithi117.front.sepia.ceph.com 2022-02-10T06:07:39.898 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T06:07:39.917 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com 2022-02-10T06:07:39.917 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T06:07:39.935 INFO:teuthology.lock.ops:Updating [smithi117.front.sepia.ceph.com]: set os type and version on server 2022-02-10T06:07:39.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2022-02-10T06:07:40.163 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2022-02-10T06:07:40.212 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Ubuntu" 2022-02-10T06:07:40.212 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2022-02-10T06:07:40.213 INFO:teuthology.orchestra.run.smithi117.stdout:ID=ubuntu 2022-02-10T06:07:40.213 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE=debian 2022-02-10T06:07:40.213 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2022-02-10T06:07:40.214 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="18.04" 2022-02-10T06:07:40.214 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-10T06:07:40.215 INFO:teuthology.orchestra.run.smithi117.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-10T06:07:40.215 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-10T06:07:40.215 INFO:teuthology.orchestra.run.smithi117.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-10T06:07:40.216 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_CODENAME=bionic 2022-02-10T06:07:40.216 INFO:teuthology.orchestra.run.smithi117.stdout:UBUNTU_CODENAME=bionic 2022-02-10T06:07:40.217 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2022-02-10T06:07:40.236 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server 2022-02-10T06:07:40.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2022-02-10T06:07:40.462 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2022-02-10T06:07:40.510 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2022-02-10T06:07:40.511 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2022-02-10T06:07:40.511 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2022-02-10T06:07:40.511 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2022-02-10T06:07:40.511 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2022-02-10T06:07:40.512 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="18.04" 2022-02-10T06:07:40.512 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-10T06:07:40.512 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-10T06:07:40.512 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-10T06:07:40.512 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-10T06:07:40.513 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=bionic 2022-02-10T06:07:40.513 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=bionic 2022-02-10T06:07:40.513 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2022-02-10T06:07:40.613 INFO:teuthology.dispatcher.supervisor:Running job 6672617 2022-02-10T06:07:40.613 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-02-10_04:55:39-orch:cephadm-wip-adk2-testing-2022-02-09-2058-pacific-distro-basic-smithi/6672617 --name adking-2022-02-10_04:55:39-orch:cephadm-wip-adk2-testing-2022-02-09-2058-pacific-distro-basic-smithi --description orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/adking-2022-02-10_04:55:39-orch:cephadm-wip-adk2-testing-2022-02-09-2058-pacific-distro-basic-smithi/6672617/orig.config.yaml 2022-02-10T06:07:40.617 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-02-10_04:55:39-orch:cephadm-wip-adk2-testing-2022-02-09-2058-pacific-distro-basic-smithi/6672617 2022-02-10T06:07:40.618 INFO:teuthology.dispatcher.supervisor:Job PID: 30192 2022-02-10T06:07:40.618 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-10T12:38:09.399 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-10T12:38:09.620 INFO:teuthology.kill:Killing Pids: {30192} 2022-02-10T12:38:10.812 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-02-10T12:38:10.813 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-02-10T12:38:10.814 INFO:teuthology.misc:Compressing logs... 2022-02-10T12:38:10.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2022-02-10T12:38:11.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2022-02-10T12:38:11.067 INFO:teuthology.orchestra.run.smithi117.stderr:find: '/var/lib/ceph/crash': No such file or directory 2022-02-10T12:38:11.358 INFO:teuthology.orchestra.run.smithi170.stderr:find: '/var/lib/ceph/crash': No such file or directory 2022-02-10T12:38:11.377 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-02-10T12:38:11.377 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2022-02-10T12:38:11.417 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-02-10T12:38:11.417 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2022-02-10T12:38:11.419 INFO:teuthology.misc:Compressing logs... 2022-02-10T12:38:11.620 INFO:teuthology.misc:Compressing logs... 2022-02-10T12:38:20.389 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-02-10T12:38:20.428 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-mon.smithi117.log: file size changed while zipping 2022-02-10T12:38:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-02-10T12:38:27.710 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-02-10T12:38:30.547 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-mon.smithi170.log: file size changed while zipping 2022-02-10T12:38:31.322 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-mgr.smithi117.lemwxl.log: file size changed while zipping 2022-02-10T12:38:36.850 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-02-10T12:38:37.702 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-02-10T12:38:44.250 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-02-10T12:38:45.571 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-02-10T12:38:52.950 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/a3446ebc-8a38-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-02-10T12:39:07.940 INFO:teuthology.kill:No teuthology processes running 2022-02-10T12:39:09.109 INFO:teuthology.kill:Nuking machines: ['smithi117', 'smithi170'] 2022-02-10T12:39:10.030 INFO:teuthology.kill:2022-02-10 12:39:10,030.030 INFO:teuthology.nuke:targets: 2022-02-10T12:39:10.031 INFO:teuthology.kill: smithi117.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrwXSaeJM7AWtNtBt+7wQDZ9L4hvLvTwhwFIAB3njFJpiXq8qBWwwwrAS//xBV6eXLlqCtqOK+Y38qcFbd2qermOszR/Q8RZIqaWwbs58gtlnc/KnzBB59tbKP/SfAa5hbnNOP3uNxsJR2fhvylJSjjOtFz/JDg7M49ZxxklirU94uIhDyJx+/3ALHfps6HUlhShQyCeJIilAGrzr+KuTq3j4CiJF1UAUj3BuNNwoPF5pMsgcADJoMAQJf6x1iHOxNxvjEgWsD6y3QLRZw5xObpkhGwGf+WNeNpz/23kzgMHZ7juWRWg2OqBYsI/f5TJZEpYeWqd4yi1FGnNGQ1c9H 2022-02-10T12:39:10.031 INFO:teuthology.kill: smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaM/fSnEMOiuTftafkAp8DZx1rXGlJQpcJXAfGjwHEIz5WFH9/uL6papqVr3cjAHmeHZSs+75i0kuHxWhOPUPDig/kp+bzIJlMLyxPSOER3MtRo757/GhK2R82cb2AsL0r2wPfZ2/cl3E1usKa3J7dC08hJJSxc/UNdcCHDcGyyMveK+KyTMtYZWoU/XV46hPH8MkcNM8SVbd3LyMLt9tp9RO//8ImtqVuZO3iqnCHrpwlly4zQ7C+uJEo3ha4Ocnm/4DuNx+YiGF1pApTUeWc3nJY5OvReCYpgyScyAXemrKwXt3xXr18IE7Xt6ZvDiLqqbAi8PK94XHcfoS+SFfl 2022-02-10T12:39:10.056 INFO:teuthology.kill:2022-02-10 12:39:10,056.056 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T12:39:10.059 INFO:teuthology.kill:2022-02-10 12:39:10,059.059 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T12:39:10.137 INFO:teuthology.kill:2022-02-10 12:39:10,137.137 INFO:teuthology.orchestra.console:Power off smithi117 2022-02-10T12:39:10.150 INFO:teuthology.kill:2022-02-10 12:39:10,150.150 INFO:teuthology.orchestra.console:Power off smithi170 2022-02-10T12:39:18.506 INFO:teuthology.kill:2022-02-10 12:39:18,506.506 INFO:teuthology.orchestra.console:Power off for smithi117 completed 2022-02-10T12:39:18.516 INFO:teuthology.kill:2022-02-10 12:39:18,515.515 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2022-02-10T12:39:18.654 INFO:teuthology.kill:2022-02-10 12:39:18,654.654 INFO:teuthology.lock.ops:unlocked: smithi117.front.sepia.ceph.com 2022-02-10T12:39:18.660 INFO:teuthology.kill:2022-02-10 12:39:18,660.660 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com 2022-02-10T12:41:18.915 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-10T12:41:19.002 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi117 but it was locked by another job: /home/teuthworker/archive/ozeneva-2022-02-10_12:18:47-rgw-wip-omri-tracing-compiled-distro-basic-smithi/6673492 2022-02-10T12:41:19.003 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi170 but it was locked by another job: /home/teuthworker/archive/ozeneva-2022-02-10_12:18:47-rgw-wip-omri-tracing-compiled-distro-basic-smithi/6673492