2022-01-07T03:05:49.559 INFO:root:teuthology version: 1.1.0-95a7d479 2022-01-07T03:05:49.682 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi145.front.sepia.ceph.com' reimaging 2022-01-07T03:05:49.684 INFO:teuthology.lock.ops:Updating [ubuntu@smithi145.front.sepia.ceph.com]: reset os type and version on server 2022-01-07T03:05:49.684 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-01-07T03:05:49.704 INFO:teuthology.lock.ops:Node 'ubuntu@smithi145.front.sepia.ceph.com' reimaging is complete 2022-01-07T03:05:49.705 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging 2022-01-07T03:05:49.705 INFO:teuthology.lock.ops:Updating [ubuntu@smithi176.front.sepia.ceph.com]: reset os type and version on server 2022-01-07T03:05:49.705 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2022-01-07T03:05:49.724 INFO:teuthology.lock.ops:Node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging is complete 2022-01-07T03:05:49.917 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of centos 8.stream 2022-01-07T03:05:49.922 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of centos 8.stream 2022-01-07T03:05:50.237 INFO:teuthology.orchestra.console:Power off smithi145 2022-01-07T03:05:50.252 INFO:teuthology.orchestra.console:Power off smithi176 2022-01-07T03:06:02.724 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-01-07T03:06:02.729 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2022-01-07T03:06:02.826 INFO:teuthology.orchestra.console:Power on smithi145 2022-01-07T03:06:02.834 INFO:teuthology.orchestra.console:Power on smithi176 2022-01-07T03:06:07.089 INFO:teuthology.orchestra.console:Power on for smithi176 completed 2022-01-07T03:06:07.190 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish 2022-01-07T03:06:11.195 INFO:teuthology.orchestra.console:Power on for smithi145 completed 2022-01-07T03:06:11.297 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2022-01-07T03:11:00.835 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-07T03:11:01.121 INFO:teuthology.provision.fog.smithi145:Node is ready 2022-01-07T03:11:01.138 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2022-01-07T03:11:01.195 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2022-01-07T03:11:01.526 INFO:teuthology.provision.fog.smithi145:Deploy complete! 2022-01-07T03:11:11.692 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-07T03:11:11.982 INFO:teuthology.provision.fog.smithi176:Node is ready 2022-01-07T03:11:11.998 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com 2022-01-07T03:11:12.055 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176 2022-01-07T03:11:12.388 INFO:teuthology.provision.fog.smithi176:Deploy complete! 2022-01-07T03:11:12.536 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com 2022-01-07T03:11:12.537 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-07T03:11:12.558 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com 2022-01-07T03:11:12.559 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-07T03:11:12.577 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server 2022-01-07T03:11:12.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-01-07T03:11:12.820 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2022-01-07T03:11:12.877 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2022-01-07T03:11:12.877 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8" 2022-01-07T03:11:12.878 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2022-01-07T03:11:12.878 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2022-01-07T03:11:12.878 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8" 2022-01-07T03:11:12.878 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2022-01-07T03:11:12.879 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-07T03:11:12.879 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2022-01-07T03:11:12.879 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-07T03:11:12.879 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2022-01-07T03:11:12.879 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-07T03:11:12.880 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-07T03:11:12.880 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-07T03:11:12.881 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-01-07T03:11:12.901 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server 2022-01-07T03:11:12.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2022-01-07T03:11:13.090 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2022-01-07T03:11:13.145 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2022-01-07T03:11:13.145 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="8" 2022-01-07T03:11:13.145 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2022-01-07T03:11:13.145 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2022-01-07T03:11:13.146 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="8" 2022-01-07T03:11:13.146 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el8" 2022-01-07T03:11:13.146 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-07T03:11:13.146 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2022-01-07T03:11:13.147 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-07T03:11:13.147 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2022-01-07T03:11:13.147 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-07T03:11:13.147 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-07T03:11:13.148 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-07T03:11:13.148 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2022-01-07T03:11:13.243 INFO:teuthology.dispatcher.supervisor:Running job 6599583 2022-01-07T03:11:13.244 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-06_15:57:04-rados-wip-yuri6-testing-2022-01-05-1255-distro-default-smithi/6599583 --name yuriw-2022-01-06_15:57:04-rados-wip-yuri6-testing-2022-01-05-1255-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} -- /home/teuthworker/archive/yuriw-2022-01-06_15:57:04-rados-wip-yuri6-testing-2022-01-05-1255-distro-default-smithi/6599583/orig.config.yaml 2022-01-07T03:11:13.247 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-06_15:57:04-rados-wip-yuri6-testing-2022-01-05-1255-distro-default-smithi/6599583 2022-01-07T03:11:13.248 INFO:teuthology.dispatcher.supervisor:Job PID: 25273 2022-01-07T03:11:13.249 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-07T15:12:07.557 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-07T15:12:07.729 INFO:teuthology.kill:Killing Pids: {25273} 2022-01-07T15:12:08.870 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-07T15:12:08.871 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-07T15:12:08.871 INFO:teuthology.misc:Compressing logs... 2022-01-07T15:12:08.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-01-07T15:12:09.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2022-01-07T15:12:09.092 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-07T15:12:09.277 INFO:teuthology.orchestra.run.smithi176.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-07T15:12:09.324 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-07T15:12:09.324 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2022-01-07T15:12:09.361 INFO:teuthology.orchestra.run.smithi176.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-07T15:12:09.362 INFO:teuthology.orchestra.run.smithi176.stderr:tar: Error is not recoverable: exiting now 2022-01-07T15:12:09.363 INFO:teuthology.misc:Compressing logs... 2022-01-07T15:12:09.903 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-01-07T15:12:10.121 INFO:teuthology.misc:Compressing logs... 2022-01-07T15:12:14.034 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-mgr.smithi145.xddpbn.log: file size changed while zipping 2022-01-07T15:12:40.080 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-07T15:12:42.859 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-07T15:13:12.584 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-07T15:13:13.551 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-07T15:13:45.439 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-07T15:13:51.742 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-07T15:14:08.709 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-mon.smithi145.log: file size changed while zipping 2022-01-07T15:14:09.082 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-07T15:14:15.175 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-mon.smithi176.log: file size changed while zipping 2022-01-07T15:14:39.689 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/ad1fa0e0-6f68-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-07T15:15:11.896 INFO:teuthology.orchestra.run.smithi145.stderr:tar: .: file changed as we read it 2022-01-07T15:15:38.206 INFO:teuthology.kill:No teuthology processes running 2022-01-07T15:15:39.338 INFO:teuthology.kill:Nuking machines: ['smithi145', 'smithi176'] 2022-01-07T15:15:40.251 INFO:teuthology.kill:2022-01-07 15:15:40,251.251 INFO:teuthology.nuke:targets: 2022-01-07T15:15:40.252 INFO:teuthology.kill: smithi145.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC/bzxzJB2zahYTvN93MF28vt3ndDp3kyPZziOrB5avakcHgRhDq43mkaaUslNyB8it6qqDIhzv1tSmiwW6dIgIR2vJEzxVpUvSPwJK0p3oVOQMN+vDsw/CP9JuKdSl8wd49cAmMCqiJYIvv/qsZ8iFfjC4/EOhElqGOh1jAke85DHd+79ETg0TI65tjYPH+ICkJYKbG+NWM3V5yQKQ1lNrW+HrO8kg8C+UZLLXL2z0TPSbvkUXJFGmXKcXgiaH/I606LIm0MENlGGKsFQoNE5C1UhSnkGKDP9DXrU0RP4iFuix3uUbOa2yBX3R+rM5mmjJ1oYTdeEXca93NSRa4rQldcKRkTSU8PljF2n7aO6IvlRJQTbpODpEsXNhrpgiB+rzpSwvdYQ62odhcUJbVfqxUkqAFmXCHlUD2Di3NvyvXY2VJlp+NRawvSG+b7lxep9MMuuzXXqxOSTIVsNfwybvXQTPHWer/MDHyutQ7pFBnFYJXk1AKMV+EtqTBo1b0c= 2022-01-07T15:15:40.253 INFO:teuthology.kill: smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD4AHt7HkVUdsveXBltnXoOYQ3BFdkGC6lw3f9W6W23vgc7yPCdJdULwcnBs8pDxf6n9FAB7FYyG/ZzLU2wNFYhLgDpnUqNoHm1dkTLAJWDrUS6xUKHNlmjvT7YsXOU3c+dyJ6x81RX8npoHDEnpZHYkRrS8nIiVBSy+V/FP9gbTbvWKC+MZrnFYg6MEVvAx+2JuxxCYAMXDpYXuvKGRKhJda8Nej8jOV+bn8JeWMnLRYY5FKTrfp5qSRWvFsBMcHGvJfAsP9pc/rUjScKoY2Xl47t0S9QARlS3hat4/TvF/7R9hYxjO7wuPcalic5hzqQDNRtByPiNagVeCcnMCruYZDvTRX0I/sWBI4dMfZjRginO6WjsGnaWmUgX6x/f28Y/BuLOYVet8M0OfWOVT3camnJirLj3OH+GhKy4PD3E3imEqcSlXC4YjHQRIeA0Jl5ketadTtU2w8+E76DjuumrgeKqdRLy4UmtzEgsAjv2lgXE6kn8+AGlE6nPelWUTM= 2022-01-07T15:15:40.278 INFO:teuthology.kill:2022-01-07 15:15:40,278.278 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-07T15:15:40.281 INFO:teuthology.kill:2022-01-07 15:15:40,281.281 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-07T15:15:40.355 INFO:teuthology.kill:2022-01-07 15:15:40,355.355 INFO:teuthology.orchestra.console:Power off smithi145 2022-01-07T15:15:40.368 INFO:teuthology.kill:2022-01-07 15:15:40,368.368 INFO:teuthology.orchestra.console:Power off smithi176 2022-01-07T15:15:48.728 INFO:teuthology.kill:2022-01-07 15:15:48,727.727 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-01-07T15:15:48.877 INFO:teuthology.kill:2022-01-07 15:15:48,876.876 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com 2022-01-07T15:15:52.843 INFO:teuthology.kill:2022-01-07 15:15:52,842.842 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2022-01-07T15:15:52.985 INFO:teuthology.kill:2022-01-07 15:15:52,985.985 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com 2022-01-07T15:17:53.197 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-07T15:17:53.304 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi145 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-07_14:22:01-upgrade:octopus-x-master-distro-default-smithi/6601329