2022-01-21T18:21:57.900 INFO:root:teuthology version: 1.1.0-75f1bcd2 2022-01-21T18:21:58.035 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi123.front.sepia.ceph.com' reimaging 2022-01-21T18:21:58.036 INFO:teuthology.lock.ops:Updating [ubuntu@smithi123.front.sepia.ceph.com]: reset os type and version on server 2022-01-21T18:21:58.037 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2022-01-21T18:21:58.056 INFO:teuthology.lock.ops:Node 'ubuntu@smithi123.front.sepia.ceph.com' reimaging is complete 2022-01-21T18:21:58.057 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging 2022-01-21T18:21:58.057 INFO:teuthology.lock.ops:Updating [ubuntu@smithi146.front.sepia.ceph.com]: reset os type and version on server 2022-01-21T18:21:58.058 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2022-01-21T18:21:58.078 INFO:teuthology.lock.ops:Node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging is complete 2022-01-21T18:21:58.272 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of rhel 8.4 2022-01-21T18:21:58.290 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of rhel 8.4 2022-01-21T18:21:58.580 INFO:teuthology.orchestra.console:Power off smithi123 2022-01-21T18:21:58.623 INFO:teuthology.orchestra.console:Power off smithi146 2022-01-21T18:22:11.077 INFO:teuthology.orchestra.console:Power off for smithi123 completed 2022-01-21T18:22:11.104 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2022-01-21T18:22:11.180 INFO:teuthology.orchestra.console:Power on smithi123 2022-01-21T18:22:11.208 INFO:teuthology.orchestra.console:Power on smithi146 2022-01-21T18:22:15.461 INFO:teuthology.orchestra.console:Power on for smithi146 completed 2022-01-21T18:22:15.562 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish 2022-01-21T18:22:19.547 INFO:teuthology.orchestra.console:Power on for smithi123 completed 2022-01-21T18:22:19.649 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish 2022-01-21T18:26:51.332 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-21T18:26:51.548 INFO:teuthology.provision.fog.smithi123:Node is ready 2022-01-21T18:26:51.563 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com 2022-01-21T18:26:51.618 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123 2022-01-21T18:26:51.936 INFO:teuthology.provision.fog.smithi123:Deploy complete! 2022-01-21T18:27:11.414 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-21T18:27:11.628 INFO:teuthology.provision.fog.smithi146:Node is ready 2022-01-21T18:27:11.643 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com 2022-01-21T18:27:11.698 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146 2022-01-21T18:27:12.028 INFO:teuthology.provision.fog.smithi146:Deploy complete! 2022-01-21T18:27:12.199 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com 2022-01-21T18:27:12.200 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-21T18:27:12.218 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com 2022-01-21T18:27:12.219 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-21T18:27:12.236 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server 2022-01-21T18:27:12.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2022-01-21T18:27:12.471 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2022-01-21T18:27:12.526 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2022-01-21T18:27:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.4 (Ootpa)" 2022-01-21T18:27:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2022-01-21T18:27:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2022-01-21T18:27:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.4" 2022-01-21T18:27:12.527 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2022-01-21T18:27:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-21T18:27:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2022-01-21T18:27:12.528 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-21T18:27:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2022-01-21T18:27:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-21T18:27:12.529 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-21T18:27:12.529 INFO:teuthology.orchestra.run.smithi123.stdout: 2022-01-21T18:27:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-21T18:27:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-21T18:27:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-21T18:27:12.530 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-21T18:27:12.531 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2022-01-21T18:27:12.549 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server 2022-01-21T18:27:12.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2022-01-21T18:27:12.769 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2022-01-21T18:27:12.823 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Red Hat Enterprise Linux" 2022-01-21T18:27:12.823 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8.4 (Ootpa)" 2022-01-21T18:27:12.824 INFO:teuthology.orchestra.run.smithi146.stdout:ID="rhel" 2022-01-21T18:27:12.824 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="fedora" 2022-01-21T18:27:12.824 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8.4" 2022-01-21T18:27:12.824 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2022-01-21T18:27:12.825 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-21T18:27:12.825 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2022-01-21T18:27:12.825 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-21T18:27:12.825 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.redhat.com/" 2022-01-21T18:27:12.826 INFO:teuthology.orchestra.run.smithi146.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-21T18:27:12.826 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-21T18:27:12.826 INFO:teuthology.orchestra.run.smithi146.stdout: 2022-01-21T18:27:12.826 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-21T18:27:12.826 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-21T18:27:12.827 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-21T18:27:12.827 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-21T18:27:12.827 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2022-01-21T18:27:12.918 INFO:teuthology.dispatcher.supervisor:Running job 6632244 2022-01-21T18:27:12.919 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_75f1bcd237a8b6022305cd53ba87e106d45aa665/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-21_15:22:24-rados-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632244 --name yuriw-2022-01-21_15:22:24-rados-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/yuriw-2022-01-21_15:22:24-rados-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632244/orig.config.yaml 2022-01-21T18:27:12.922 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-21_15:22:24-rados-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632244 2022-01-21T18:27:12.923 INFO:teuthology.dispatcher.supervisor:Job PID: 4543 2022-01-21T18:27:12.924 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-22T00:57:41.229 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-22T00:57:41.382 INFO:teuthology.kill:Killing Pids: {4543} 2022-01-22T00:57:42.504 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-22T00:57:42.506 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-22T00:57:42.506 INFO:teuthology.misc:Compressing logs... 2022-01-22T00:57:42.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2022-01-22T00:57:42.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2022-01-22T00:57:42.743 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-22T00:57:42.983 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-22T00:57:43.025 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-22T00:57:43.026 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2022-01-22T00:57:43.080 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-22T00:57:43.081 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2022-01-22T00:57:43.082 INFO:teuthology.misc:Compressing logs... 2022-01-22T00:57:43.255 INFO:teuthology.misc:Compressing logs... 2022-01-22T00:57:50.714 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-22T00:57:52.718 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-22T00:57:56.172 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-mon.smithi146.log: file size changed while zipping 2022-01-22T00:57:58.177 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-22T00:57:59.912 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-mgr.smithi123.bzqhhl.log: file size changed while zipping 2022-01-22T00:58:03.309 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-22T00:58:08.163 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-22T00:58:12.754 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-22T00:58:18.031 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-22T00:58:20.029 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-22T00:58:27.519 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/3c6980aa-7ae9-11ec-8c35-001a4aab830c/ceph-mon.smithi123.log: file size changed while zipping 2022-01-22T00:58:44.473 INFO:teuthology.kill:No teuthology processes running 2022-01-22T00:58:45.863 INFO:teuthology.kill:Nuking machines: ['smithi123', 'smithi146'] 2022-01-22T00:58:46.829 INFO:teuthology.kill:2022-01-22 00:58:46,828.828 INFO:teuthology.nuke:targets: 2022-01-22T00:58:46.830 INFO:teuthology.kill: smithi123.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3FIvxDsiFk7w0X2U1DHHche27pn2G/scSRS7rGFhv+1PDCaNAyIt43LKSqXD9nm8fsH76zL/tYRg9Q4GHlbz0zUSGQVfOsLhHmlVY9AYAN+MTFSTzuey4B3OG34U8EQ3pwtHJe0l3yU8yJ/bxGNiNMOQB7N8dAmGVbYr5WxLk4Wd7ysH1JQQl49nSS7ky8RKR8i16ilCOMCLMsoRdGs5HBAOxq+MGtrz3f2wayDWHbcmVYq93DzCMmcehZpLfru5StLZmuO/q2Z9d+QbpItfGfA1hHfnIOrrHxrHfFz4iSbuRKZrCFxrqoygewbeQ3TtJun5Vpb2YOhJAWTthd+Vy6tXkbhovxx+nNr38bAmtarYYp4Sd4mWiv3VoB37jjHmdOHvfUgHGCxSBO9Hrl6yYq2mXMu2EwqeD7hOWspokFJQu38bnZeokeO5gZ7FUVt/Fvms4Vm/d6+sr575uN97sqYTnWyzpi36gGhcz3uDaL/UxUDh23qAHS+w5LNhX3ks= 2022-01-22T00:58:46.830 INFO:teuthology.kill: smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhoDypxPQ5LP1cT9/3TZYYior+4rDx5N1b9kLFZYJiarohH9uCZIVWkYbxqEJlBp0T65j2PB+kNhjyQemero0Yrv7ZE30qT+YTef8on0fz5zTWll40GeJ6RO6L6kOiO2Plye4D3Pnqj1zihqv5PZMBWJO9wLH1taKlAHj2RO7E8uONsZGYSfmgAxIdxIqjCKtcEYVdpILNJeEGv8wTuzjDNyGY6EJasCCwHLBb/dbT4tbgedVl8D5gpodYtdcsWeJwYbMgrs9sEYJh1fsUdoRAPeHZn3EBsvF3V05utgm+Kts0Nsd7OHRDZpndJj/0cfyWwu7xOfNagN6XJ0LYEvZxZoyAh1tOhXU6i0tcMg6tGNXYpYJTCuFiPwzfHOX/8eG4wURxFx6z79gMJLUIU3V7EgssMwS7zWp3zhcwjX/9xkK/OKZls4FQJVeDvJlBZNNKrbjiI8Z9bi2ezy7gyjKtWzQFzo9WvM8dBOS0gf13sCWNo/rlWg8kocBBPi0IGF0= 2022-01-22T00:58:46.859 INFO:teuthology.kill:2022-01-22 00:58:46,858.858 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-22T00:58:46.862 INFO:teuthology.kill:2022-01-22 00:58:46,862.862 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-22T00:58:46.939 INFO:teuthology.kill:2022-01-22 00:58:46,938.938 INFO:teuthology.orchestra.console:Power off smithi146 2022-01-22T00:58:46.952 INFO:teuthology.kill:2022-01-22 00:58:46,951.951 INFO:teuthology.orchestra.console:Power off smithi123 2022-01-22T00:58:59.422 INFO:teuthology.kill:2022-01-22 00:58:59,422.422 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2022-01-22T00:58:59.427 INFO:teuthology.kill:2022-01-22 00:58:59,427.427 INFO:teuthology.orchestra.console:Power off for smithi123 completed 2022-01-22T00:58:59.577 INFO:teuthology.kill:2022-01-22 00:58:59,576.576 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com 2022-01-22T00:58:59.578 INFO:teuthology.kill:2022-01-22 00:58:59,578.578 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com 2022-01-22T01:00:59.890 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-22T01:00:59.975 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi123 but it was locked by another job: /home/teuthworker/archive/dis-2022-01-21_20:35:25-rbd-wip-dis-testing-distro-default-smithi/6633295 2022-01-22T01:00:59.976 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi146 but it was locked by another job: /home/teuthworker/archive/dis-2022-01-21_20:35:25-rbd-wip-dis-testing-distro-default-smithi/6633295