2022-01-24T22:17:42.831 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-24T22:17:42.952 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi067.front.sepia.ceph.com' reimaging 2022-01-24T22:17:42.953 INFO:teuthology.lock.ops:Updating [ubuntu@smithi067.front.sepia.ceph.com]: reset os type and version on server 2022-01-24T22:17:42.953 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2022-01-24T22:17:42.971 INFO:teuthology.lock.ops:Node 'ubuntu@smithi067.front.sepia.ceph.com' reimaging is complete 2022-01-24T22:17:42.971 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi178.front.sepia.ceph.com' reimaging 2022-01-24T22:17:42.972 INFO:teuthology.lock.ops:Updating [ubuntu@smithi178.front.sepia.ceph.com]: reset os type and version on server 2022-01-24T22:17:42.972 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2022-01-24T22:17:42.989 INFO:teuthology.lock.ops:Node 'ubuntu@smithi178.front.sepia.ceph.com' reimaging is complete 2022-01-24T22:17:43.184 INFO:teuthology.provision.fog.smithi067:Scheduling deploy of centos 8.stream 2022-01-24T22:17:43.193 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of centos 8.stream 2022-01-24T22:17:43.502 INFO:teuthology.orchestra.console:Power off smithi178 2022-01-24T22:17:43.515 INFO:teuthology.orchestra.console:Power off smithi067 2022-01-24T22:17:51.880 INFO:teuthology.orchestra.console:Power off for smithi067 completed 2022-01-24T22:17:51.885 INFO:teuthology.orchestra.console:Power off for smithi178 completed 2022-01-24T22:17:51.981 INFO:teuthology.orchestra.console:Power on smithi067 2022-01-24T22:17:51.988 INFO:teuthology.orchestra.console:Power on smithi178 2022-01-24T22:18:00.347 INFO:teuthology.orchestra.console:Power on for smithi067 completed 2022-01-24T22:18:00.353 INFO:teuthology.orchestra.console:Power on for smithi178 completed 2022-01-24T22:18:00.449 INFO:teuthology.provision.fog.smithi067:Waiting for deploy to finish 2022-01-24T22:18:00.454 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish 2022-01-24T22:22:41.568 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-24T22:22:41.839 INFO:teuthology.provision.fog.smithi067:Node is ready 2022-01-24T22:22:41.857 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067.front.sepia.ceph.com 2022-01-24T22:22:41.915 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67 smithi067.front.sepia.ceph.com smithi067 2022-01-24T22:22:42.270 INFO:teuthology.provision.fog.smithi067:Deploy complete! 2022-01-24T22:23:05.619 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-24T22:23:05.862 INFO:teuthology.provision.fog.smithi178:Node is ready 2022-01-24T22:23:05.879 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com 2022-01-24T22:23:05.937 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178 2022-01-24T22:23:06.271 INFO:teuthology.provision.fog.smithi178:Deploy complete! 2022-01-24T22:23:06.448 INFO:teuthology.lock.ops:Checking smithi067.front.sepia.ceph.com 2022-01-24T22:23:06.449 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-24T22:23:06.469 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com 2022-01-24T22:23:06.469 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-24T22:23:06.488 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: set os type and version on server 2022-01-24T22:23:06.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2022-01-24T22:23:06.741 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2022-01-24T22:23:06.799 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2022-01-24T22:23:06.799 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8" 2022-01-24T22:23:06.799 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2022-01-24T22:23:06.800 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2022-01-24T22:23:06.800 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8" 2022-01-24T22:23:06.800 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8" 2022-01-24T22:23:06.800 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-24T22:23:06.801 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2022-01-24T22:23:06.801 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-24T22:23:06.801 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2022-01-24T22:23:06.801 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-24T22:23:06.801 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-24T22:23:06.802 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-24T22:23:06.803 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2022-01-24T22:23:06.821 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server 2022-01-24T22:23:06.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2022-01-24T22:23:07.007 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2022-01-24T22:23:07.063 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2022-01-24T22:23:07.064 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8" 2022-01-24T22:23:07.065 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2022-01-24T22:23:07.065 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2022-01-24T22:23:07.065 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8" 2022-01-24T22:23:07.065 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2022-01-24T22:23:07.066 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-24T22:23:07.066 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2022-01-24T22:23:07.066 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-24T22:23:07.067 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2022-01-24T22:23:07.067 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-24T22:23:07.067 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-24T22:23:07.067 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-24T22:23:07.068 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2022-01-24T22:23:07.164 INFO:teuthology.dispatcher.supervisor:Running job 6637940 2022-01-24T22:23:07.165 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_swagner@teuthology --archive /home/teuthworker/archive/swagner-2022-01-24_16:47:41-orch:cephadm-wip-swagner-testing-2022-01-24-1121-distro-default-smithi/6637940 --name swagner-2022-01-24_16:47:41-orch:cephadm-wip-swagner-testing-2022-01-24-1121-distro-default-smithi --description orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} -- /home/teuthworker/archive/swagner-2022-01-24_16:47:41-orch:cephadm-wip-swagner-testing-2022-01-24-1121-distro-default-smithi/6637940/orig.config.yaml 2022-01-24T22:23:07.169 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/swagner-2022-01-24_16:47:41-orch:cephadm-wip-swagner-testing-2022-01-24-1121-distro-default-smithi/6637940 2022-01-24T22:23:07.170 INFO:teuthology.dispatcher.supervisor:Job PID: 12789 2022-01-24T22:23:07.171 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-25T04:53:36.784 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-25T04:53:37.007 INFO:teuthology.kill:Killing Pids: {12789} 2022-01-25T04:53:38.482 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-25T04:53:38.484 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-25T04:53:38.484 INFO:teuthology.misc:Compressing logs... 2022-01-25T04:53:38.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2022-01-25T04:53:38.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2022-01-25T04:53:38.759 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-25T04:53:38.955 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-25T04:53:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-25T04:53:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2022-01-25T04:53:39.025 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-25T04:53:39.025 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2022-01-25T04:53:39.026 INFO:teuthology.misc:Compressing logs... 2022-01-25T04:53:39.428 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-01-25T04:53:39.600 INFO:teuthology.misc:Compressing logs... 2022-01-25T04:53:41.787 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-mon.smithi067.log: file size changed while zipping 2022-01-25T04:53:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-mgr.smithi067.pqcuux.log: file size changed while zipping 2022-01-25T04:53:43.598 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-25T04:53:45.705 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-25T04:53:46.878 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-25T04:53:49.221 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-25T04:53:50.951 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-25T04:53:53.240 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-25T04:53:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-25T04:53:59.512 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /var/log/ceph/585a93d4-7d65-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-25T04:54:31.657 INFO:teuthology.orchestra.run.smithi178.stderr:tar: .: file changed as we read it 2022-01-25T04:54:31.714 INFO:teuthology.kill:No teuthology processes running 2022-01-25T04:54:32.865 INFO:teuthology.kill:Nuking machines: ['smithi067', 'smithi178'] 2022-01-25T04:54:33.887 INFO:teuthology.kill:2022-01-25 04:54:33,886.886 INFO:teuthology.nuke:targets: 2022-01-25T04:54:33.888 INFO:teuthology.kill: smithi067.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/Q2yxYQLegvyskPzwHcFn+JytZSCTU6cxbV/+Z63L0tKITXF/lKfW4XkAwZeenqldfrCPj2dJoOBtAG13qWdY5y1DTkTDKTARpBmkCCske61P2CzNryVpz+bYxlWlYTuUpqApkm+3rVc38HtGFp7wq2m37fKQABkgqRaLkuQEZFE+B3eK5vslssf8RtnoWEbcrhSRyLCDij2vjuem3s2UhyEXipAN4qgHW3vyxhF+Pka1qzNF4RhLpppFGZR8CDjpsKdJcUgZdx32sjtn2v2FdJLOtgri1/9T/qtxCdvV2fTuoRsLqiVIDRT60766YrcXyRR4x3ffxPtAmtRy2hR/2hDucqB37KRUHfzqF8lg2BuAiqSrvH1aKRE6VwYJ3E654tKh6PbkaAIxcy8MZWWxS53j06LVMfl5krbMLeyJ3mw7WJ1e0I0L1n4Wk2Zp+SQ7Cf2P/tQ665FSRtGXxtZhLvzH3GcFO0+gvt0YbnWIqIuX4DU0XWfkoMo4toErSJU= 2022-01-25T04:54:33.888 INFO:teuthology.kill: smithi178.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuyuWmuHarPHOTxqjh7sz8EXK4gpUZbclWJls2v/8MAW/bK5mc/zK85CXvwRuab65M2CxGNB7JK6N0fZfN6yzvWzSiOiUVcuZlfNTB4ZYU1nLuSiYs8ZkGp/cITujNvXhLKK/S0RHkJt49+55W/XEgZ1Vd9amC/TBmfzoZT3FK6HHth0kwG+/szDVy8g0Db/PPbkmtgiKe+j6Y9qVJLXwR0eUNm5NBdU5Xa+/QWmRmjuTde/MlQ9bsDgjkmI2Tq/hw38nun/vc+K00O8HkG33ezgfVgSJLui22p+1NXItzZzojqEIL4PFaMSN5Cn+ivoeIEreziW3lm2z3AmzXx4cEaeHatDFxFL0mWOOn/RXADIOWwzC+rF4CKpJbKcSD1ifEOQhpvpqhwB8kwlbD3IWb3nRRU69O7cLQ4vBuFbWJrBWD4v8IUMpi2i98PL8qtxpojMShl1R1fjeuGaQcE2mwialFrmDF7+lSb4AqHYUwrehi7mMZP2NXXC1Tql0Y+gs= 2022-01-25T04:54:33.919 INFO:teuthology.kill:2022-01-25 04:54:33,919.919 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-25T04:54:33.924 INFO:teuthology.kill:2022-01-25 04:54:33,924.924 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-25T04:54:34.006 INFO:teuthology.kill:2022-01-25 04:54:34,006.006 INFO:teuthology.orchestra.console:Power off smithi067 2022-01-25T04:54:34.023 INFO:teuthology.kill:2022-01-25 04:54:34,022.022 INFO:teuthology.orchestra.console:Power off smithi178 2022-01-25T04:54:42.378 INFO:teuthology.kill:2022-01-25 04:54:42,378.378 INFO:teuthology.orchestra.console:Power off for smithi067 completed 2022-01-25T04:54:42.525 INFO:teuthology.kill:2022-01-25 04:54:42,524.524 INFO:teuthology.lock.ops:unlocked: smithi067.front.sepia.ceph.com 2022-01-25T04:54:46.497 INFO:teuthology.kill:2022-01-25 04:54:46,496.496 INFO:teuthology.orchestra.console:Power off for smithi178 completed 2022-01-25T04:54:46.645 INFO:teuthology.kill:2022-01-25 04:54:46,645.645 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com 2022-01-25T04:56:47.004 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-25T04:56:47.112 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi067 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-25_03:40:28-rados-wip-yuri4-testing-2022-01-24-1706-pacific-distro-default-smithi/6639148 2022-01-25T04:56:47.113 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi178 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-25_03:40:28-rados-wip-yuri4-testing-2022-01-24-1706-pacific-distro-default-smithi/6639147