2022-02-20T13:09:40.081 INFO:root:teuthology version: 1.1.0-eea25212 2022-02-20T13:09:40.204 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging 2022-02-20T13:09:40.206 INFO:teuthology.lock.ops:Updating [ubuntu@smithi052.front.sepia.ceph.com]: reset os type and version on server 2022-02-20T13:09:40.206 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2022-02-20T13:09:40.225 INFO:teuthology.lock.ops:Node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging is complete 2022-02-20T13:09:40.225 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi066.front.sepia.ceph.com' reimaging 2022-02-20T13:09:40.225 INFO:teuthology.lock.ops:Updating [ubuntu@smithi066.front.sepia.ceph.com]: reset os type and version on server 2022-02-20T13:09:40.226 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2022-02-20T13:09:40.245 INFO:teuthology.lock.ops:Node 'ubuntu@smithi066.front.sepia.ceph.com' reimaging is complete 2022-02-20T13:09:40.440 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of ubuntu 18.04 2022-02-20T13:09:40.464 INFO:teuthology.provision.fog.smithi066:Scheduling deploy of ubuntu 18.04 2022-02-20T13:09:40.878 INFO:teuthology.orchestra.console:Power off smithi052 2022-02-20T13:09:41.021 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2022-02-20T13:09:41.030 INFO:teuthology.orchestra.console:Power off smithi066 2022-02-20T13:09:41.123 INFO:teuthology.orchestra.console:Power on smithi052 2022-02-20T13:09:41.180 INFO:teuthology.orchestra.console:Power off for smithi066 completed 2022-02-20T13:09:41.281 INFO:teuthology.orchestra.console:Power on smithi066 2022-02-20T13:09:45.380 INFO:teuthology.orchestra.console:Power on for smithi052 completed 2022-02-20T13:09:45.482 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish 2022-02-20T13:09:49.647 INFO:teuthology.orchestra.console:Power on for smithi066 completed 2022-02-20T13:09:49.748 INFO:teuthology.provision.fog.smithi066:Waiting for deploy to finish 2022-02-20T13:14:23.682 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-20T13:14:23.952 INFO:teuthology.provision.fog.smithi052:Node is ready 2022-02-20T13:14:24.000 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052.front.sepia.ceph.com 2022-02-20T13:14:24.053 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52 smithi052.front.sepia.ceph.com smithi052 2022-02-20T13:14:24.336 INFO:teuthology.provision.fog.smithi052:Deploy complete! 2022-02-20T13:15:44.842 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-20T13:15:50.861 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-20T13:15:56.875 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-20T13:16:02.890 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-20T13:16:08.905 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-02-20T13:16:14.993 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-20T13:16:15.268 INFO:teuthology.provision.fog.smithi066:Node is ready 2022-02-20T13:16:15.318 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066.front.sepia.ceph.com 2022-02-20T13:16:15.367 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66 smithi066.front.sepia.ceph.com smithi066 2022-02-20T13:16:15.670 INFO:teuthology.provision.fog.smithi066:Deploy complete! 2022-02-20T13:16:15.844 INFO:teuthology.lock.ops:Checking smithi052.front.sepia.ceph.com 2022-02-20T13:16:15.845 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-20T13:16:15.866 INFO:teuthology.lock.ops:Checking smithi066.front.sepia.ceph.com 2022-02-20T13:16:15.867 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-20T13:16:15.887 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: set os type and version on server 2022-02-20T13:16:15.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2022-02-20T13:16:16.182 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2022-02-20T13:16:16.233 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="Ubuntu" 2022-02-20T13:16:16.233 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2022-02-20T13:16:16.234 INFO:teuthology.orchestra.run.smithi052.stdout:ID=ubuntu 2022-02-20T13:16:16.234 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE=debian 2022-02-20T13:16:16.234 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2022-02-20T13:16:16.234 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="18.04" 2022-02-20T13:16:16.235 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-20T13:16:16.235 INFO:teuthology.orchestra.run.smithi052.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-20T13:16:16.235 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-20T13:16:16.235 INFO:teuthology.orchestra.run.smithi052.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-20T13:16:16.236 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_CODENAME=bionic 2022-02-20T13:16:16.236 INFO:teuthology.orchestra.run.smithi052.stdout:UBUNTU_CODENAME=bionic 2022-02-20T13:16:16.237 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2022-02-20T13:16:16.256 INFO:teuthology.lock.ops:Updating [smithi066.front.sepia.ceph.com]: set os type and version on server 2022-02-20T13:16:16.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2022-02-20T13:16:16.497 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2022-02-20T13:16:16.546 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="Ubuntu" 2022-02-20T13:16:16.546 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2022-02-20T13:16:16.547 INFO:teuthology.orchestra.run.smithi066.stdout:ID=ubuntu 2022-02-20T13:16:16.547 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE=debian 2022-02-20T13:16:16.547 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2022-02-20T13:16:16.547 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="18.04" 2022-02-20T13:16:16.548 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-20T13:16:16.548 INFO:teuthology.orchestra.run.smithi066.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-20T13:16:16.548 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-20T13:16:16.548 INFO:teuthology.orchestra.run.smithi066.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-20T13:16:16.549 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_CODENAME=bionic 2022-02-20T13:16:16.549 INFO:teuthology.orchestra.run.smithi066.stdout:UBUNTU_CODENAME=bionic 2022-02-20T13:16:16.549 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2022-02-20T13:16:16.644 INFO:teuthology.dispatcher.supervisor:Running job 6693073 2022-02-20T13:16:16.645 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-02-17_23:23:56-rados-wip-yuri7-testing-2022-02-17-0852-pacific-distro-default-smithi/6693073 --name yuriw-2022-02-17_23:23:56-rados-wip-yuri7-testing-2022-02-17-0852-pacific-distro-default-smithi --description rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} -- /home/teuthworker/archive/yuriw-2022-02-17_23:23:56-rados-wip-yuri7-testing-2022-02-17-0852-pacific-distro-default-smithi/6693073/orig.config.yaml 2022-02-20T13:16:16.648 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-02-17_23:23:56-rados-wip-yuri7-testing-2022-02-17-0852-pacific-distro-default-smithi/6693073 2022-02-20T13:16:16.648 INFO:teuthology.dispatcher.supervisor:Job PID: 22424 2022-02-20T13:16:16.649 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-20T19:46:45.879 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-20T19:46:46.066 INFO:teuthology.kill:Killing Pids: {22424} 2022-02-20T19:46:47.205 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-02-20T19:46:47.206 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-02-20T19:46:47.207 INFO:teuthology.misc:Compressing logs... 2022-02-20T19:46:47.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2022-02-20T19:46:47.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2022-02-20T19:46:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:find: '/var/lib/ceph/crash': No such file or directory 2022-02-20T19:46:47.760 INFO:teuthology.orchestra.run.smithi066.stderr:find: '/var/lib/ceph/crash': No such file or directory 2022-02-20T19:46:47.783 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-02-20T19:46:47.783 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now 2022-02-20T19:46:47.819 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-02-20T19:46:47.820 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2022-02-20T19:46:47.823 INFO:teuthology.misc:Compressing logs... 2022-02-20T19:46:48.074 INFO:teuthology.misc:Compressing logs... 2022-02-20T19:46:48.205 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-mgr.smithi066.tycdpb.log: file size changed while zipping 2022-02-20T19:46:50.309 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-02-20T19:46:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-02-20T19:46:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-02-20T19:46:53.928 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-mon.smithi066.log: file size changed while zipping 2022-02-20T19:46:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-mgr.smithi052.bsmjyq.log: file size changed while zipping 2022-02-20T19:46:55.793 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-02-20T19:46:57.683 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-02-20T19:46:59.528 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-02-20T19:47:04.736 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-mon.smithi052.log: file size changed while zipping 2022-02-20T19:47:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/3d8015b6-9250-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-02-20T19:47:13.023 INFO:teuthology.kill:No teuthology processes running 2022-02-20T19:47:14.154 INFO:teuthology.kill:Nuking machines: ['smithi052', 'smithi066'] 2022-02-20T19:47:15.112 INFO:teuthology.kill:2022-02-20 19:47:15,112.112 INFO:teuthology.nuke:targets: 2022-02-20T19:47:15.113 INFO:teuthology.kill: smithi052.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDTfZW9Yh5lAow04AqizM+SFTTK7qwOyl20q4sDGFLL15KyMZdJQ/VXVlflAmoz4acEUCI195n2n5QY+r8nQ1O9+FlUXQ43zauTK9wcOlRuUk6WeEG1yOOWZ532zHWdMWiEYy2UeWyxt+rtTFwa6sz/jOdRyUPDv3v2LBCV+lPiGfrPXQhies85D6H5idXWvmbTJhEWz5wBCf9Eg98/p+DLSRrk7k2JmamEp0V5SjE0EoKL2vL0OFrT1QQoaYPMwY5EhVvgNRTTeQfPx/mCzcAbzURNciQspXa0ijEbyB1nxXzsCsE8TcvTkySV03Y/5spFZ8GHnbyjRmiI1oOHyzsX 2022-02-20T19:47:15.113 INFO:teuthology.kill: smithi066.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIgDy9h4OQXgYmBFAkXEE+X0IDjcbVJiFWbfILmi/JZHRwGDehmvMnC1BXhqSVsxWk8rwkEy+DrbM3xJxAeXRX/jjDt4WIiJbtJsPvN5E6r2hEsEFpPTdmU499wIIsqFeDcWES4t2mod7n3cnEK3fyhPqRFl+j7n9YuQ+hCfQAwza2tAKgRXYPkIigpV9AcDkCSQeaZhT+n5JtZ/7DJAVe9WhU50YDyoA8oy91bQR3MCBBaBpfnRmlPxdJMZYNGCyGUc2lsnJZLzw9nVy8JmdsDss5gygSbBq0PMjOmg3gpy6pwmvwbX75M51Q0P4WbRODf+tcCbL70TB11VDVcbPl 2022-02-20T19:47:15.142 INFO:teuthology.kill:2022-02-20 19:47:15,140.140 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-20T19:47:15.144 INFO:teuthology.kill:2022-02-20 19:47:15,144.144 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-20T19:47:15.218 INFO:teuthology.kill:2022-02-20 19:47:15,218.218 INFO:teuthology.orchestra.console:Power off smithi052 2022-02-20T19:47:15.232 INFO:teuthology.kill:2022-02-20 19:47:15,232.232 INFO:teuthology.orchestra.console:Power off smithi066 2022-02-20T19:47:23.589 INFO:teuthology.kill:2022-02-20 19:47:23,588.588 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2022-02-20T19:47:23.594 INFO:teuthology.kill:2022-02-20 19:47:23,594.594 INFO:teuthology.orchestra.console:Power off for smithi066 completed 2022-02-20T19:47:23.738 INFO:teuthology.kill:2022-02-20 19:47:23,738.738 INFO:teuthology.lock.ops:unlocked: smithi066.front.sepia.ceph.com 2022-02-20T19:47:23.743 INFO:teuthology.kill:2022-02-20 19:47:23,743.743 INFO:teuthology.lock.ops:unlocked: smithi052.front.sepia.ceph.com 2022-02-20T19:49:24.067 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-20T19:49:24.153 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi052 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-02-15_16:52:18-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6685616 2022-02-20T19:49:24.154 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi066 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-02-15_16:52:18-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6685616