2022-04-06T19:25:54.369 INFO:root:teuthology version: 1.1.0-3dd79763 2022-04-06T19:25:54.491 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi100.front.sepia.ceph.com' reimaging 2022-04-06T19:25:54.492 INFO:teuthology.lock.ops:Updating [ubuntu@smithi100.front.sepia.ceph.com]: reset os type and version on server 2022-04-06T19:25:54.493 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2022-04-06T19:25:54.510 INFO:teuthology.lock.ops:Node 'ubuntu@smithi100.front.sepia.ceph.com' reimaging is complete 2022-04-06T19:25:54.511 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi163.front.sepia.ceph.com' reimaging 2022-04-06T19:25:54.511 INFO:teuthology.lock.ops:Updating [ubuntu@smithi163.front.sepia.ceph.com]: reset os type and version on server 2022-04-06T19:25:54.511 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2022-04-06T19:25:54.529 INFO:teuthology.lock.ops:Node 'ubuntu@smithi163.front.sepia.ceph.com' reimaging is complete 2022-04-06T19:25:54.723 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of ubuntu 18.04 2022-04-06T19:25:54.728 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of ubuntu 18.04 2022-04-06T19:25:55.042 INFO:teuthology.orchestra.console:Power off smithi163 2022-04-06T19:25:55.056 INFO:teuthology.orchestra.console:Power off smithi100 2022-04-06T19:26:03.415 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2022-04-06T19:26:03.420 INFO:teuthology.orchestra.console:Power off for smithi100 completed 2022-04-06T19:26:03.516 INFO:teuthology.orchestra.console:Power on smithi163 2022-04-06T19:26:03.523 INFO:teuthology.orchestra.console:Power on smithi100 2022-04-06T19:26:11.930 INFO:teuthology.orchestra.console:Power on for smithi163 completed 2022-04-06T19:26:11.969 INFO:teuthology.orchestra.console:Power on for smithi100 completed 2022-04-06T19:26:12.031 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish 2022-04-06T19:26:12.069 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish 2022-04-06T19:28:58.261 ERROR:teuthology.orchestra.connection:Error authenticating with smithi100.front.sepia.ceph.com: Authentication failed. 2022-04-06T19:31:03.069 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-06T19:31:03.320 INFO:teuthology.provision.fog.smithi100:Node is ready 2022-04-06T19:31:03.368 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com 2022-04-06T19:31:03.416 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100 2022-04-06T19:31:03.732 INFO:teuthology.provision.fog.smithi100:Deploy complete! 2022-04-06T19:31:53.126 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-06T19:31:53.412 INFO:teuthology.provision.fog.smithi163:Node is ready 2022-04-06T19:31:53.462 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163.front.sepia.ceph.com 2022-04-06T19:31:53.511 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163 2022-04-06T19:31:53.846 INFO:teuthology.provision.fog.smithi163:Deploy complete! 2022-04-06T19:31:54.021 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com 2022-04-06T19:31:54.022 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-06T19:31:54.042 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com 2022-04-06T19:31:54.042 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-06T19:31:54.062 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server 2022-04-06T19:31:54.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2022-04-06T19:31:54.372 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2022-04-06T19:31:54.425 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu" 2022-04-06T19:31:54.425 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2022-04-06T19:31:54.426 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu 2022-04-06T19:31:54.426 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian 2022-04-06T19:31:54.427 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2022-04-06T19:31:54.427 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="18.04" 2022-04-06T19:31:54.428 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/" 2022-04-06T19:31:54.428 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-04-06T19:31:54.430 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-04-06T19:31:54.430 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-04-06T19:31:54.431 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=bionic 2022-04-06T19:31:54.431 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=bionic 2022-04-06T19:31:54.434 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2022-04-06T19:31:54.457 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server 2022-04-06T19:31:54.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2022-04-06T19:31:54.750 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2022-04-06T19:31:54.799 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Ubuntu" 2022-04-06T19:31:54.799 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2022-04-06T19:31:54.800 INFO:teuthology.orchestra.run.smithi163.stdout:ID=ubuntu 2022-04-06T19:31:54.800 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE=debian 2022-04-06T19:31:54.801 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2022-04-06T19:31:54.801 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="18.04" 2022-04-06T19:31:54.801 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.ubuntu.com/" 2022-04-06T19:31:54.802 INFO:teuthology.orchestra.run.smithi163.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-04-06T19:31:54.802 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-04-06T19:31:54.803 INFO:teuthology.orchestra.run.smithi163.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-04-06T19:31:54.803 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_CODENAME=bionic 2022-04-06T19:31:54.803 INFO:teuthology.orchestra.run.smithi163.stdout:UBUNTU_CODENAME=bionic 2022-04-06T19:31:54.804 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2022-04-06T19:31:54.897 INFO:teuthology.dispatcher.supervisor:Running job 6779669 2022-04-06T19:31:54.897 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3dd797633f37e2d7e2c4ae3ef9ac35a188c2028d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-04-06_14:02:46-rados-wip-yuri4-testing-2022-04-05-1720-pacific-distro-default-smithi/6779669 --name yuriw-2022-04-06_14:02:46-rados-wip-yuri4-testing-2022-04-05-1720-pacific-distro-default-smithi --description rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} -- /home/teuthworker/archive/yuriw-2022-04-06_14:02:46-rados-wip-yuri4-testing-2022-04-05-1720-pacific-distro-default-smithi/6779669/orig.config.yaml 2022-04-06T19:31:54.900 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-04-06_14:02:46-rados-wip-yuri4-testing-2022-04-05-1720-pacific-distro-default-smithi/6779669 2022-04-06T19:31:54.901 INFO:teuthology.dispatcher.supervisor:Job PID: 13819 2022-04-06T19:31:54.902 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-04-07T02:02:05.649 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-04-07T02:02:05.800 INFO:teuthology.kill:Killing Pids: {13819} 2022-04-07T02:02:06.984 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-04-07T02:02:06.986 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-04-07T02:02:06.986 INFO:teuthology.misc:Compressing logs... 2022-04-07T02:02:06.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2022-04-07T02:02:07.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2022-04-07T02:02:07.251 INFO:teuthology.orchestra.run.smithi100.stderr:find: '/var/lib/ceph/crash': No such file or directory 2022-04-07T02:02:07.546 INFO:teuthology.orchestra.run.smithi163.stderr:find: '/var/lib/ceph/crash': No such file or directory 2022-04-07T02:02:07.566 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-04-07T02:02:07.567 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2022-04-07T02:02:07.611 INFO:teuthology.orchestra.run.smithi163.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-04-07T02:02:07.611 INFO:teuthology.orchestra.run.smithi163.stderr:tar: Error is not recoverable: exiting now 2022-04-07T02:02:07.613 INFO:teuthology.misc:Compressing logs... 2022-04-07T02:02:07.853 INFO:teuthology.misc:Compressing logs... 2022-04-07T02:02:09.802 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-mgr.smithi100.rojiud.log: file size changed while zipping 2022-04-07T02:02:11.389 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-mon.smithi163.log: file size changed while zipping 2022-04-07T02:02:13.117 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-04-07T02:02:15.144 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-04-07T02:02:16.895 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-04-07T02:02:18.652 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-04-07T02:02:19.026 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-mon.smithi100.log: file size changed while zipping 2022-04-07T02:02:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-04-07T02:02:23.183 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-04-07T02:02:24.945 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/a5ce0232-b5e0-11ec-8c36-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-04-07T02:02:31.126 INFO:teuthology.kill:No teuthology processes running 2022-04-07T02:02:32.253 INFO:teuthology.kill:Nuking machines: ['smithi100', 'smithi163'] 2022-04-07T02:02:33.246 INFO:teuthology.kill:2022-04-07 02:02:33,245.245 INFO:teuthology.nuke:targets: 2022-04-07T02:02:33.247 INFO:teuthology.kill: smithi100.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCywPJnQO1IiqigTru4NopHz3mPqiawbr/XdQbTyNvX9FBtdT/LX2bvkgYhFubOM+/cJiip7Cq+BrSFg//L7Bv8DAmHwR4BPaaqyBpy5S6Y0j+/5cC5qo1foFbTqtbe+ezfQYoGDnqm6dVog9WuFUcAHkMGEC2cwP5VOPvuYHeprU4xbDED7Ve3gVOh0+v0eN9ffCsV1UUTrsI0dygIC4W/Hf9K4mtkj+HekpLbaKvJ9UKzW+fTCB6kdrIGI0mbDF8TM2t+ye/Z3fdTQkpBa+aCFiHeQB5dMLzSWWaequpuRBPnWN2IstSw+zNepSppLiEYReihaxBCZXwWyKIBI72B 2022-04-07T02:02:33.247 INFO:teuthology.kill: smithi163.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCwoEdFIqEQykgN73aHeJGhLfdfWyBLwjr8mhJTp0wm5Mj58PprNVqV2mRMR0VAdYW7pr0LUSWmn7rmMAj2vnlFdqc6TIePh7qr2RLqEgPyioh0X7Hjy62ri6ZV9qYuqwh2cXixDLvaDwS3zoBe3IA4qIRVWiT4ojgqzHBrUyJhgGwEIykmamCvv1TzZ1zHFju/paBKYkn6VhKBFXQArTy7oSK8d2wMRtQg++YVKjgEbXjIoJf5uThfBR56SK/DmSQCNQPTvrnOa4wrnbfYgMM21R3x4nQfMMxe3jTQTYdCBrWrS8QnhhSoOhOBSdgVuyG271FEco3udu4qM58fSdN/ 2022-04-07T02:02:33.272 INFO:teuthology.kill:2022-04-07 02:02:33,271.271 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-07T02:02:33.274 INFO:teuthology.kill:2022-04-07 02:02:33,274.274 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-07T02:02:33.347 INFO:teuthology.kill:2022-04-07 02:02:33,347.347 INFO:teuthology.orchestra.console:Power off smithi163 2022-04-07T02:02:33.361 INFO:teuthology.kill:2022-04-07 02:02:33,361.361 INFO:teuthology.orchestra.console:Power off smithi100 2022-04-07T02:02:41.717 INFO:teuthology.kill:2022-04-07 02:02:41,716.716 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2022-04-07T02:02:41.724 INFO:teuthology.kill:2022-04-07 02:02:41,724.724 INFO:teuthology.orchestra.console:Power off for smithi100 completed 2022-04-07T02:02:41.864 INFO:teuthology.kill:2022-04-07 02:02:41,863.863 INFO:teuthology.lock.ops:unlocked: smithi163.front.sepia.ceph.com 2022-04-07T02:02:41.867 INFO:teuthology.kill:2022-04-07 02:02:41,866.866 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com 2022-04-07T02:04:42.078 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-04-07T02:04:42.161 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi100 but it was locked by another job: /home/teuthworker/archive/adking-2022-04-06_23:59:43-orch:cephadm-wip-adk2-testing-2022-04-06-1631-distro-basic-smithi/6780275 2022-04-07T02:04:42.161 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi163 but it was locked by another job: /home/teuthworker/archive/adking-2022-04-06_23:59:43-orch:cephadm-wip-adk2-testing-2022-04-06-1631-distro-basic-smithi/6780275