2022-01-27T17:17:04.781 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-27T17:17:04.911 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging 2022-01-27T17:17:04.912 INFO:teuthology.lock.ops:Updating [ubuntu@smithi059.front.sepia.ceph.com]: reset os type and version on server 2022-01-27T17:17:04.913 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-01-27T17:17:04.938 INFO:teuthology.lock.ops:Node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging is complete 2022-01-27T17:17:04.938 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging 2022-01-27T17:17:04.939 INFO:teuthology.lock.ops:Updating [ubuntu@smithi187.front.sepia.ceph.com]: reset os type and version on server 2022-01-27T17:17:04.939 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2022-01-27T17:17:04.962 INFO:teuthology.lock.ops:Node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging is complete 2022-01-27T17:17:05.185 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of rhel 8.4 2022-01-27T17:17:05.191 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of rhel 8.4 2022-01-27T17:17:05.569 INFO:teuthology.orchestra.console:Power off smithi059 2022-01-27T17:17:05.605 INFO:teuthology.orchestra.console:Power off smithi187 2022-01-27T17:17:05.746 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2022-01-27T17:17:05.848 INFO:teuthology.orchestra.console:Power on smithi187 2022-01-27T17:17:10.103 INFO:teuthology.orchestra.console:Power on for smithi187 completed 2022-01-27T17:17:10.204 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish 2022-01-27T17:17:18.056 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-01-27T17:17:18.157 INFO:teuthology.orchestra.console:Power on smithi059 2022-01-27T17:17:22.424 INFO:teuthology.orchestra.console:Power on for smithi059 completed 2022-01-27T17:17:22.525 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2022-01-27T17:20:38.724 ERROR:teuthology.orchestra.connection:Error authenticating with smithi059.front.sepia.ceph.com: Authentication timeout. 2022-01-27T17:21:10.203 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-27T17:21:20.436 INFO:teuthology.provision.fog.smithi059:Node is ready 2022-01-27T17:21:20.456 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2022-01-27T17:21:20.514 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2022-01-27T17:21:20.854 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2022-01-27T17:21:29.547 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-01-27T17:21:41.822 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-27T17:21:42.023 INFO:teuthology.provision.fog.smithi187:Node is ready 2022-01-27T17:21:42.080 INFO:teuthology.orchestra.run.smithi187.stdout:smithi187.front.sepia.ceph.com 2022-01-27T17:21:42.136 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.187 smithi187.front.sepia.ceph.com smithi187 2022-01-27T17:21:42.458 INFO:teuthology.provision.fog.smithi187:Deploy complete! 2022-01-27T17:21:42.606 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2022-01-27T17:21:42.607 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-27T17:21:42.627 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com 2022-01-27T17:21:42.629 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-27T17:21:42.666 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2022-01-27T17:21:42.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-01-27T17:21:42.871 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2022-01-27T17:21:42.928 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2022-01-27T17:21:42.929 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.4 (Ootpa)" 2022-01-27T17:21:42.929 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2022-01-27T17:21:42.929 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2022-01-27T17:21:42.930 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.4" 2022-01-27T17:21:42.930 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2022-01-27T17:21:42.930 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-27T17:21:42.930 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2022-01-27T17:21:42.931 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-27T17:21:42.931 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2022-01-27T17:21:42.931 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-27T17:21:42.931 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-27T17:21:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: 2022-01-27T17:21:42.932 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-27T17:21:42.932 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-27T17:21:42.933 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-27T17:21:42.933 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-27T17:21:42.935 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-01-27T17:21:42.951 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server 2022-01-27T17:21:42.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2022-01-27T17:21:43.135 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2022-01-27T17:21:43.190 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux" 2022-01-27T17:21:43.191 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.4 (Ootpa)" 2022-01-27T17:21:43.191 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel" 2022-01-27T17:21:43.191 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora" 2022-01-27T17:21:43.191 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.4" 2022-01-27T17:21:43.192 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2022-01-27T17:21:43.192 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-27T17:21:43.192 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2022-01-27T17:21:43.192 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-27T17:21:43.193 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/" 2022-01-27T17:21:43.193 INFO:teuthology.orchestra.run.smithi187.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-27T17:21:43.194 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-27T17:21:43.194 INFO:teuthology.orchestra.run.smithi187.stdout: 2022-01-27T17:21:43.194 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-27T17:21:43.195 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-27T17:21:43.195 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-27T17:21:43.195 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-27T17:21:43.196 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2022-01-27T17:21:43.298 INFO:teuthology.dispatcher.supervisor:Running job 6644465 2022-01-27T17:21:43.298 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-27_17:06:11-rados-wip-yuri4-testing-2022-01-26-0857-quincy-distro-default-smithi/6644465 --name yuriw-2022-01-27_17:06:11-rados-wip-yuri4-testing-2022-01-26-0857-quincy-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} -- /home/teuthworker/archive/yuriw-2022-01-27_17:06:11-rados-wip-yuri4-testing-2022-01-26-0857-quincy-distro-default-smithi/6644465/orig.config.yaml 2022-01-27T17:21:43.301 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-27_17:06:11-rados-wip-yuri4-testing-2022-01-26-0857-quincy-distro-default-smithi/6644465 2022-01-27T17:21:43.302 INFO:teuthology.dispatcher.supervisor:Job PID: 29732 2022-01-27T17:21:43.303 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-27T23:51:54.049 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-27T23:51:54.232 INFO:teuthology.kill:Killing Pids: {29732} 2022-01-27T23:51:55.393 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-27T23:51:55.394 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-27T23:51:55.395 INFO:teuthology.misc:Compressing logs... 2022-01-27T23:51:55.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-01-27T23:51:55.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2022-01-27T23:51:55.631 INFO:teuthology.orchestra.run.smithi059.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-27T23:51:55.868 INFO:teuthology.orchestra.run.smithi187.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-27T23:51:55.920 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-27T23:51:55.920 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2022-01-27T23:51:55.964 INFO:teuthology.orchestra.run.smithi187.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-27T23:51:55.965 INFO:teuthology.orchestra.run.smithi187.stderr:tar: Error is not recoverable: exiting now 2022-01-27T23:51:55.966 INFO:teuthology.misc:Compressing logs... 2022-01-27T23:51:56.167 INFO:teuthology.misc:Compressing logs... 2022-01-27T23:52:03.797 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-27T23:52:05.035 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-27T23:52:11.464 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-27T23:52:11.805 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-27T23:52:19.868 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-27T23:52:20.528 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-27T23:52:23.926 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-mon.smithi187.log: file size changed while zipping 2022-01-27T23:52:30.059 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-27T23:52:31.009 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-27T23:52:39.841 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-mon.smithi059.log: file size changed while zipping 2022-01-27T23:52:41.710 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/1ff5cc00-7f97-11ec-8c35-001a4aab830c/ceph-mgr.smithi059.sycpyf.log: file size changed while zipping 2022-01-27T23:52:58.576 INFO:teuthology.kill:No teuthology processes running 2022-01-27T23:52:59.854 INFO:teuthology.kill:Nuking machines: ['smithi059', 'smithi187'] 2022-01-27T23:53:00.840 INFO:teuthology.kill:2022-01-27 23:53:00,840.840 INFO:teuthology.nuke:targets: 2022-01-27T23:53:00.841 INFO:teuthology.kill: smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpQ9Yxgl2ZzY03cfoE8V5O9NQoh3ptC7n+kte5vYY9v5nhn+36zePiTWE6ujY3zyox5G8tnKfiU7E+47o8I6k2rSSpnz/gHM4juv9dyunq5U8EecD4moZpbeLZ/dpUfsacmLVpoyeMxYCBjLlKwiQQjXx7rDcR0zmq8aNFj91l6lmJhgM5AxwOKtlCrsfruSbmhGs4fHESJhCEozvjB0tRWZ0Ujb/Oqqvs/AH4UWgKtXHyaKe/Z1EnRnPgKIVv5kZR3PrlQfjGZdQ/6pjbxe9p7WGShafxXhE+CaEp97kERO6oKPKYsFAYFV6gXyU3VVy0xj6bp3+TpTiaiKRVGx3usYwVsjhkbvAVEsMoPwXnkzkGKBQgGCdHgml/tYxQhVZ97gmKPylXC4LJbvlsO7yx+42ES7OLaYdS5wjblJkwxchn/jOTFuQyQIK/4/FwDdkmCJshCASOJLY19Z4AOvr2oG/6yahVHY0k/ciI2SSSTPHoVEy11VNCv3l3NdOXLBc= 2022-01-27T23:53:00.842 INFO:teuthology.kill: smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCn8Rt3ezNEEv3mVFA/zG/8JCiSt2cOKXFPd3p99ifxw84Scrx2/PjRixMv0yWBaJblXGpHLDpoRs0yqJ4zTS6C4ZjjavBusF+qrywIJle3csX5cDQ9VYsK5+kYAQdCjZwyriJ0RjnOhG1QggFfntzBYDo/CZz2aSArXipmfnz4oP7mETlTniId/3768K33UM4kJbANGyruYwOnTt7Y5gGsU8j0B7lMCm9NKTJTu77SOETz0pIYVLiTrfOaxw9m1bew+q5f2GPARtuxT8X8AL09xFIjY2U8xQsOkeO4W1jDkmo0AFxsPrX6wGYAUIITMAsSI7pnJevcba9EQHgJaVRj7HQuIrXrlSlZuF3Dsf3VTXbibCq53VKrQ3yjYg+a7gxghl1h2BzppavMP8R+Q74ZuAbMfUNysBdKmTLTUAeRYBamjvn83GcsmVLL3vK6Li8KdRtqDGNW9ZxcF+zw9sxZJY3c0Twy3Whj20jHXrtu30WP1jCX8TY2OFtWlzaSUXk= 2022-01-27T23:53:00.868 INFO:teuthology.kill:2022-01-27 23:53:00,867.867 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-27T23:53:00.872 INFO:teuthology.kill:2022-01-27 23:53:00,871.871 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-27T23:53:00.956 INFO:teuthology.kill:2022-01-27 23:53:00,956.956 INFO:teuthology.orchestra.console:Power off smithi187 2022-01-27T23:53:00.972 INFO:teuthology.kill:2022-01-27 23:53:00,972.972 INFO:teuthology.orchestra.console:Power off smithi059 2022-01-27T23:53:09.337 INFO:teuthology.kill:2022-01-27 23:53:09,337.337 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2022-01-27T23:53:09.342 INFO:teuthology.kill:2022-01-27 23:53:09,342.342 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-01-27T23:53:09.484 INFO:teuthology.kill:2022-01-27 23:53:09,483.483 INFO:teuthology.lock.ops:unlocked: smithi187.front.sepia.ceph.com 2022-01-27T23:53:09.487 INFO:teuthology.kill:2022-01-27 23:53:09,487.487 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2022-01-27T23:55:09.798 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-27T23:55:09.891 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi059 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-27_15:09:25-rados-wip-yuri6-testing-2022-01-26-1547-distro-default-smithi/6643857