2022-01-26T23:29:45.171 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-26T23:29:45.295 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi073.front.sepia.ceph.com' reimaging 2022-01-26T23:29:45.297 INFO:teuthology.lock.ops:Updating [ubuntu@smithi073.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T23:29:45.297 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2022-01-26T23:29:45.319 INFO:teuthology.lock.ops:Node 'ubuntu@smithi073.front.sepia.ceph.com' reimaging is complete 2022-01-26T23:29:45.319 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging 2022-01-26T23:29:45.320 INFO:teuthology.lock.ops:Updating [ubuntu@smithi150.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T23:29:45.320 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2022-01-26T23:29:45.341 INFO:teuthology.lock.ops:Node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging is complete 2022-01-26T23:29:45.537 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of centos 8.3 2022-01-26T23:29:45.564 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of centos 8.3 2022-01-26T23:29:45.906 INFO:teuthology.orchestra.console:Power off smithi150 2022-01-26T23:29:45.923 INFO:teuthology.orchestra.console:Power off smithi073 2022-01-26T23:29:54.284 INFO:teuthology.orchestra.console:Power off for smithi073 completed 2022-01-26T23:29:54.385 INFO:teuthology.orchestra.console:Power on smithi073 2022-01-26T23:29:58.386 INFO:teuthology.orchestra.console:Power off for smithi150 completed 2022-01-26T23:29:58.488 INFO:teuthology.orchestra.console:Power on smithi150 2022-01-26T23:30:02.740 INFO:teuthology.orchestra.console:Power on for smithi150 completed 2022-01-26T23:30:02.752 INFO:teuthology.orchestra.console:Power on for smithi073 completed 2022-01-26T23:30:02.842 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish 2022-01-26T23:30:02.853 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish 2022-01-26T23:34:33.393 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T23:34:43.658 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T23:34:43.964 INFO:teuthology.provision.fog.smithi073:Node is ready 2022-01-26T23:34:43.982 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073.front.sepia.ceph.com 2022-01-26T23:34:44.038 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73 smithi073.front.sepia.ceph.com smithi073 2022-01-26T23:34:44.389 INFO:teuthology.provision.fog.smithi073:Deploy complete! 2022-01-26T23:34:53.414 INFO:teuthology.provision.fog.smithi150:Node is ready 2022-01-26T23:34:53.433 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com 2022-01-26T23:34:53.489 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150 2022-01-26T23:34:53.848 INFO:teuthology.provision.fog.smithi150:Deploy complete! 2022-01-26T23:34:53.993 INFO:teuthology.lock.ops:Checking smithi073.front.sepia.ceph.com 2022-01-26T23:34:53.994 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T23:34:54.014 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com 2022-01-26T23:34:54.014 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T23:34:54.032 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: set os type and version on server 2022-01-26T23:34:54.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2022-01-26T23:34:54.300 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2022-01-26T23:34:54.353 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Linux" 2022-01-26T23:34:54.354 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8" 2022-01-26T23:34:54.354 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2022-01-26T23:34:54.354 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2022-01-26T23:34:54.354 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8" 2022-01-26T23:34:54.355 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8" 2022-01-26T23:34:54.355 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-26T23:34:54.355 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2022-01-26T23:34:54.355 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-26T23:34:54.356 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2022-01-26T23:34:54.356 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-26T23:34:54.356 INFO:teuthology.orchestra.run.smithi073.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-26T23:34:54.356 INFO:teuthology.orchestra.run.smithi073.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-26T23:34:54.358 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2022-01-26T23:34:54.376 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server 2022-01-26T23:34:54.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2022-01-26T23:34:54.636 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2022-01-26T23:34:54.689 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Linux" 2022-01-26T23:34:54.690 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8" 2022-01-26T23:34:54.690 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2022-01-26T23:34:54.690 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2022-01-26T23:34:54.691 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8" 2022-01-26T23:34:54.691 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2022-01-26T23:34:54.691 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-26T23:34:54.691 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2022-01-26T23:34:54.692 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-26T23:34:54.692 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2022-01-26T23:34:54.692 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-26T23:34:54.692 INFO:teuthology.orchestra.run.smithi150.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-26T23:34:54.692 INFO:teuthology.orchestra.run.smithi150.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-26T23:34:54.693 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2022-01-26T23:34:54.824 INFO:teuthology.dispatcher.supervisor:Running job 6642053 2022-01-26T23:34:54.824 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-26_19:23:01-rados-wip-yuri4-testing-2022-01-26-0857-quincy-distro-default-smithi/6642053 --name yuriw-2022-01-26_19:23:01-rados-wip-yuri4-testing-2022-01-26-0857-quincy-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} -- /home/teuthworker/archive/yuriw-2022-01-26_19:23:01-rados-wip-yuri4-testing-2022-01-26-0857-quincy-distro-default-smithi/6642053/orig.config.yaml 2022-01-26T23:34:54.827 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-26_19:23:01-rados-wip-yuri4-testing-2022-01-26-0857-quincy-distro-default-smithi/6642053 2022-01-26T23:34:54.828 INFO:teuthology.dispatcher.supervisor:Job PID: 13820 2022-01-26T23:34:54.828 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-27T06:05:23.687 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-27T06:05:23.868 INFO:teuthology.kill:Killing Pids: {13820} 2022-01-27T06:05:25.039 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-27T06:05:25.040 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-27T06:05:25.040 INFO:teuthology.misc:Compressing logs... 2022-01-27T06:05:25.041 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2022-01-27T06:05:25.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2022-01-27T06:05:25.278 INFO:teuthology.orchestra.run.smithi073.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-27T06:05:25.553 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-27T06:05:25.592 INFO:teuthology.orchestra.run.smithi073.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-27T06:05:25.592 INFO:teuthology.orchestra.run.smithi073.stderr:tar: Error is not recoverable: exiting now 2022-01-27T06:05:25.632 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-27T06:05:25.632 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2022-01-27T06:05:25.634 INFO:teuthology.misc:Compressing logs... 2022-01-27T06:05:25.849 INFO:teuthology.misc:Compressing logs... 2022-01-27T06:05:29.186 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-mon.smithi150.log: file size changed while zipping 2022-01-27T06:05:40.671 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-27T06:05:44.896 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-27T06:06:00.842 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-27T06:06:01.536 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-27T06:06:13.074 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-27T06:06:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-27T06:06:29.757 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-27T06:06:29.818 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-client.rgw.foo.smithi150.yojyey.log: file size changed while zipping 2022-01-27T06:06:31.161 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-mon.smithi073.log: file size changed while zipping 2022-01-27T06:06:33.323 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-mgr.smithi073.tdfjnm.log: file size changed while zipping 2022-01-27T06:06:50.447 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/d3713ec2-7f01-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-27T06:07:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:tar: .: file changed as we read it 2022-01-27T06:07:21.989 INFO:teuthology.kill:No teuthology processes running 2022-01-27T06:07:23.105 INFO:teuthology.kill:Nuking machines: ['smithi073', 'smithi150'] 2022-01-27T06:07:24.107 INFO:teuthology.kill:2022-01-27 06:07:24,107.107 INFO:teuthology.nuke:targets: 2022-01-27T06:07:24.108 INFO:teuthology.kill: smithi073.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD477p02rIzbyDLK4qURqSMuJ2+yMgiv7JiVA3nWuQSYGQmOg/eMumVHK4OOeYEPm9QkSRin1kmVuLtMTJrqg7z0K5YnYk0pxbOzkgTLDX7mroE2GVfjAVnftznmy6EmBkY38a0NkMJR9qF9NHisnuy3f+kT8E+4PUUqIAS7KioxaQO0TUJpnY7G3jFZR7nFNTT3CmPbO14M/4PZyfG2uXmm6F0MJno+SieVpyvFE5APnOtDMXl2uoUF+i++Mr9yiKyOB/L62Cc977f8+vP2N8GSqsgGYvwltp88kLTdR1qYWTmSMwJ32T/qgkHeJKlWETpUoeASoJ+NooQTlghUHjygouBouAC2PfSHwhB7BEHJ8kLwT7zocHa2R6Qsg45sXspTNzkhyg6q/m/2q6N7euDxuwa7PyBk1ct37NvvR7HhWOGEAGURLoTQ+i+MCZT0dvoRCf5bIlq7XDfbm/E/Dhdoq91WFqoPMXL/Xb41cHNf9qA41MRSF+LRIu00R06Aok= 2022-01-27T06:07:24.108 INFO:teuthology.kill: smithi150.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRpgU70fNlgAKo7PZU58UncYo89LBZWfjyodSw3Dl5075HLGy+twVOs7yOhPiHj1I7F4coEyn7BQz7hqoYJwVRKUBwED3RenEV756S/lHhIRsn+YsT4vNffe6eGIam9RNb0DSVZ2MiBgQVPPtSKpTyr3s6cpuuhcLBttfqPy0/86ZxvsUJSaY6mfOZ1xqref6a8cvqW7K8P8lmlfh18GWusjF7bm7KZgKp/IhFMmy7m+Rxh9z/ZCEK1uiZ7SfbKAJ0yB7WXEB4kPciNeHGjepmjKPE3IUDhZdw/flwdRPGJaMSg/BUEjyTr50CaZS98Vl6d1JsKLj4jVRU6L4zyipbZ5/MtdlBdOT7RTP1ZR3LJFQRfc5BLC1mLqlC8SHxQ7IkS9cOjm2onpC72O9T9T/YEsqYFOEKKJQcTgReZJHn5KHPOewqk+g+Wx6wCkULFIzXTFHyuyETRpuuC0kt546sW0U8pW6kAK2UQhofxne4Ru5cPgkjxpLbApLnJUzfB8U= 2022-01-27T06:07:24.133 INFO:teuthology.kill:2022-01-27 06:07:24,133.133 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-27T06:07:24.137 INFO:teuthology.kill:2022-01-27 06:07:24,136.136 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-27T06:07:24.212 INFO:teuthology.kill:2022-01-27 06:07:24,212.212 INFO:teuthology.orchestra.console:Power off smithi073 2022-01-27T06:07:24.226 INFO:teuthology.kill:2022-01-27 06:07:24,225.225 INFO:teuthology.orchestra.console:Power off smithi150 2022-01-27T06:07:32.589 INFO:teuthology.kill:2022-01-27 06:07:32,588.588 INFO:teuthology.orchestra.console:Power off for smithi150 completed 2022-01-27T06:07:32.742 INFO:teuthology.kill:2022-01-27 06:07:32,741.741 INFO:teuthology.lock.ops:unlocked: smithi150.front.sepia.ceph.com 2022-01-27T06:07:36.697 INFO:teuthology.kill:2022-01-27 06:07:36,697.697 INFO:teuthology.orchestra.console:Power off for smithi073 completed 2022-01-27T06:07:36.845 INFO:teuthology.kill:2022-01-27 06:07:36,845.845 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com 2022-01-27T06:09:37.174 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-27T06:09:37.271 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi073 but it was locked by another job: /home/teuthworker/archive/mgfritch-2022-01-27_02:08:43-orch:cephadm-wip-mgfritch-testing-2022-01-26-1549-distro-default-smithi/6642505 2022-01-27T06:09:37.272 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi150 but it was locked by another job: /home/teuthworker/archive/mgfritch-2022-01-27_02:08:43-orch:cephadm-wip-mgfritch-testing-2022-01-26-1549-distro-default-smithi/6642507