2022-01-15T13:29:51.769 INFO:root:teuthology version: 1.1.0-3ff86a5d 2022-01-15T13:29:51.900 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging 2022-01-15T13:29:51.901 INFO:teuthology.lock.ops:Updating [ubuntu@smithi164.front.sepia.ceph.com]: reset os type and version on server 2022-01-15T13:29:51.901 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2022-01-15T13:29:51.919 INFO:teuthology.lock.ops:Node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging is complete 2022-01-15T13:29:51.920 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi189.front.sepia.ceph.com' reimaging 2022-01-15T13:29:51.920 INFO:teuthology.lock.ops:Updating [ubuntu@smithi189.front.sepia.ceph.com]: reset os type and version on server 2022-01-15T13:29:51.920 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2022-01-15T13:29:51.939 INFO:teuthology.lock.ops:Node 'ubuntu@smithi189.front.sepia.ceph.com' reimaging is complete 2022-01-15T13:29:52.130 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of rhel 8.4 2022-01-15T13:29:52.142 INFO:teuthology.provision.fog.smithi189:Scheduling deploy of rhel 8.4 2022-01-15T13:29:52.497 INFO:teuthology.orchestra.console:Power off smithi164 2022-01-15T13:29:52.551 INFO:teuthology.orchestra.console:Power off smithi189 2022-01-15T13:30:00.922 INFO:teuthology.orchestra.console:Power off for smithi189 completed 2022-01-15T13:30:01.024 INFO:teuthology.orchestra.console:Power on smithi189 2022-01-15T13:30:04.998 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2022-01-15T13:30:05.100 INFO:teuthology.orchestra.console:Power on smithi164 2022-01-15T13:30:09.389 INFO:teuthology.orchestra.console:Power on for smithi189 completed 2022-01-15T13:30:09.490 INFO:teuthology.provision.fog.smithi189:Waiting for deploy to finish 2022-01-15T13:30:13.467 INFO:teuthology.orchestra.console:Power on for smithi164 completed 2022-01-15T13:30:13.569 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2022-01-15T13:35:14.541 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-15T13:35:14.778 INFO:teuthology.provision.fog.smithi189:Node is ready 2022-01-15T13:35:14.795 INFO:teuthology.orchestra.run.smithi189.stdout:smithi189.front.sepia.ceph.com 2022-01-15T13:35:14.851 INFO:teuthology.orchestra.run.smithi189.stdout:172.21.15.189 smithi189.front.sepia.ceph.com smithi189 2022-01-15T13:35:15.183 INFO:teuthology.provision.fog.smithi189:Deploy complete! 2022-01-15T13:35:18.697 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-15T13:35:18.962 INFO:teuthology.provision.fog.smithi164:Node is ready 2022-01-15T13:35:18.978 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2022-01-15T13:35:19.035 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2022-01-15T13:35:19.362 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2022-01-15T13:35:19.510 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2022-01-15T13:35:19.511 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-15T13:35:19.530 INFO:teuthology.lock.ops:Checking smithi189.front.sepia.ceph.com 2022-01-15T13:35:19.531 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-15T13:35:19.550 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2022-01-15T13:35:19.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-01-15T13:35:19.779 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2022-01-15T13:35:19.833 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Red Hat Enterprise Linux" 2022-01-15T13:35:19.834 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8.4 (Ootpa)" 2022-01-15T13:35:19.834 INFO:teuthology.orchestra.run.smithi164.stdout:ID="rhel" 2022-01-15T13:35:19.834 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="fedora" 2022-01-15T13:35:19.835 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8.4" 2022-01-15T13:35:19.835 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2022-01-15T13:35:19.835 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-15T13:35:19.835 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2022-01-15T13:35:19.836 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-15T13:35:19.836 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.redhat.com/" 2022-01-15T13:35:19.836 INFO:teuthology.orchestra.run.smithi164.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-15T13:35:19.836 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-15T13:35:19.837 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-01-15T13:35:19.837 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-15T13:35:19.837 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-15T13:35:19.837 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-15T13:35:19.837 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-15T13:35:19.839 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2022-01-15T13:35:19.858 INFO:teuthology.lock.ops:Updating [smithi189.front.sepia.ceph.com]: set os type and version on server 2022-01-15T13:35:19.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2022-01-15T13:35:20.093 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2022-01-15T13:35:20.149 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="Red Hat Enterprise Linux" 2022-01-15T13:35:20.149 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="8.4 (Ootpa)" 2022-01-15T13:35:20.149 INFO:teuthology.orchestra.run.smithi189.stdout:ID="rhel" 2022-01-15T13:35:20.150 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="fedora" 2022-01-15T13:35:20.150 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="8.4" 2022-01-15T13:35:20.150 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el8" 2022-01-15T13:35:20.150 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-15T13:35:20.151 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2022-01-15T13:35:20.151 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-15T13:35:20.151 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://www.redhat.com/" 2022-01-15T13:35:20.151 INFO:teuthology.orchestra.run.smithi189.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-15T13:35:20.151 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-15T13:35:20.152 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-01-15T13:35:20.152 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-15T13:35:20.152 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-15T13:35:20.152 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-15T13:35:20.153 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-15T13:35:20.153 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2022-01-15T13:35:20.245 INFO:teuthology.dispatcher.supervisor:Running job 6619572 2022-01-15T13:35:20.246 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3ff86a5dbaafecdb6414fa2fea970bb44e836829/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619572 --name yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} -- /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619572/orig.config.yaml 2022-01-15T13:35:20.249 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619572 2022-01-15T13:35:20.250 INFO:teuthology.dispatcher.supervisor:Job PID: 22522 2022-01-15T13:35:20.250 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-15T20:05:49.367 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-15T20:05:49.521 INFO:teuthology.kill:Killing Pids: {22522} 2022-01-15T20:05:50.835 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-15T20:05:50.836 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-15T20:05:50.836 INFO:teuthology.misc:Compressing logs... 2022-01-15T20:05:50.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-01-15T20:05:51.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2022-01-15T20:05:51.072 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-15T20:05:51.259 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-15T20:05:51.298 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-15T20:05:51.299 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2022-01-15T20:05:51.429 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-15T20:05:51.430 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2022-01-15T20:05:51.431 INFO:teuthology.misc:Compressing logs... 2022-01-15T20:05:51.602 INFO:teuthology.misc:Compressing logs... 2022-01-15T20:05:58.867 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-15T20:06:00.627 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-mgr.smithi164.dzwaqq.log: file size changed while zipping 2022-01-15T20:06:00.748 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-15T20:06:04.195 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-mon.smithi189.log: file size changed while zipping 2022-01-15T20:06:07.887 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-15T20:06:11.295 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-15T20:06:15.884 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-15T20:06:20.190 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-15T20:06:25.666 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-15T20:06:27.464 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-15T20:06:34.830 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/71f94532-7609-11ec-8c32-001a4aab830c/ceph-mon.smithi164.log: file size changed while zipping 2022-01-15T20:06:51.607 INFO:teuthology.kill:No teuthology processes running 2022-01-15T20:06:52.759 INFO:teuthology.kill:Nuking machines: ['smithi164', 'smithi189'] 2022-01-15T20:06:53.745 INFO:teuthology.kill:2022-01-15 20:06:53,745.745 INFO:teuthology.nuke:targets: 2022-01-15T20:06:53.746 INFO:teuthology.kill: smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpnIPNzloLsf3GgJ2SVwh+ln44+LSJFqTLXQtEMt82REesSbg5XjDX1F6Fm5O0/vZ6aJxTP6eidxpeak1CSMWwAbmsgUYC2hKvOT+IzhknhltfWATJuokxjLQbQ32nPneUf1DxSfiUBhp57xmUa/Cw22WJ/t+ZtgFxve/AwIO+/USlGh38LTyN9Pi2IoncUbq+TciMtsbY1Vk9YkZoKyu/88dZg4/c3BA8+yS3bV6xoG8BhzkSy4pgCZv2SCZJQlIvLZq+uCf3OEnY2IzjDrdklf+HIjxfqGnqgiJW89Ne3HSZS2RR78xzJhty3U7opFKMRp3YPk0Sfz8XrWTvNp8cD6zb5kQTtVON1P3h5ai9qANP/LIVAOiHbNeBPCb1KYV7Ukbc2OmP4ifUnMaYNo7pmqYg+obmic1VUJYzxsAypUuiOl8fdGg9EKtP03fTegw2JnFJek21ds+X4ok1cOS8TPI7N2MvkvWzukNMFQJOrqXl6w5ZFOjYWmgO/7pWLKc= 2022-01-15T20:06:53.747 INFO:teuthology.kill: smithi189.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpmfI8LpAH29INp4Wxs8iXrxLx5KBvhonNZxVk3Zswu8YOl2VGGhzzUCH+OE40zJHlB7aaUhQTJbwcb6vlgvM8NE6XSId4V1Ui+4fWTdY1g3p9Q7XCGsERwXHE3xcwecTYccXdBp+dn/jLENKn48WwSpG36LGgAXuYgrGa1HaqggQxcl7OpQlgRTSYYTFWdwIkHo5KEA9hrFM/9Kwi5vzr4KePzpqZd4ENM44mFxv3cCQBYCzMwZFTNvetE2hEAxSpALPvW7n1DJcnxGaYFLbMcWkVnrUC/2GzlE/Hg/EVz43ehXfzU9HGRaiROpLTNFPe6FW8wv2WERS2U4Cj6WoY8JOHju75djj6VIfr+mXgtfo790w3yHjWaCoAFGnAvkqagwOHtCWbhn2NkDqWL/bwb4svbdjmvSnp6aiaZEmhlvJ1oDm1l/niT5FSBXh1S2Ji/x7nSg+ahJx8kjvM2WcZdusTcRSFlAYYesEI75QNnZiecYPbG7bzYRu0lo7kiRc= 2022-01-15T20:06:53.774 INFO:teuthology.kill:2022-01-15 20:06:53,773.773 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-15T20:06:53.778 INFO:teuthology.kill:2022-01-15 20:06:53,778.778 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-15T20:06:53.856 INFO:teuthology.kill:2022-01-15 20:06:53,856.856 INFO:teuthology.orchestra.console:Power off smithi189 2022-01-15T20:06:53.869 INFO:teuthology.kill:2022-01-15 20:06:53,869.869 INFO:teuthology.orchestra.console:Power off smithi164 2022-01-15T20:07:02.230 INFO:teuthology.kill:2022-01-15 20:07:02,230.230 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2022-01-15T20:07:02.391 INFO:teuthology.kill:2022-01-15 20:07:02,390.390 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com 2022-01-15T20:07:06.335 INFO:teuthology.kill:2022-01-15 20:07:06,334.334 INFO:teuthology.orchestra.console:Power off for smithi189 completed 2022-01-15T20:07:06.481 INFO:teuthology.kill:2022-01-15 20:07:06,481.481 INFO:teuthology.lock.ops:unlocked: smithi189.front.sepia.ceph.com 2022-01-15T20:09:06.794 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-15T20:09:06.883 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi164 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-15_15:17:31-rgw-wip-yuriw-quincy-1.14.22-distro-default-smithi/6619941