2022-01-15T11:48:53.969 INFO:root:teuthology version: 1.1.0-3ff86a5d 2022-01-15T11:48:54.107 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi022.front.sepia.ceph.com' reimaging 2022-01-15T11:48:54.109 INFO:teuthology.lock.ops:Updating [ubuntu@smithi022.front.sepia.ceph.com]: reset os type and version on server 2022-01-15T11:48:54.109 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2022-01-15T11:48:54.128 INFO:teuthology.lock.ops:Node 'ubuntu@smithi022.front.sepia.ceph.com' reimaging is complete 2022-01-15T11:48:54.128 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging 2022-01-15T11:48:54.128 INFO:teuthology.lock.ops:Updating [ubuntu@smithi114.front.sepia.ceph.com]: reset os type and version on server 2022-01-15T11:48:54.129 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2022-01-15T11:48:54.155 INFO:teuthology.lock.ops:Node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging is complete 2022-01-15T11:48:54.348 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of centos 8.2 2022-01-15T11:48:54.364 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of centos 8.2 2022-01-15T11:48:54.674 INFO:teuthology.orchestra.console:Power off smithi022 2022-01-15T11:48:54.721 INFO:teuthology.orchestra.console:Power off smithi114 2022-01-15T11:49:03.058 INFO:teuthology.orchestra.console:Power off for smithi022 completed 2022-01-15T11:49:03.088 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2022-01-15T11:49:03.160 INFO:teuthology.orchestra.console:Power on smithi022 2022-01-15T11:49:03.190 INFO:teuthology.orchestra.console:Power on smithi114 2022-01-15T11:49:11.529 INFO:teuthology.orchestra.console:Power on for smithi022 completed 2022-01-15T11:49:11.558 INFO:teuthology.orchestra.console:Power on for smithi114 completed 2022-01-15T11:49:11.631 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish 2022-01-15T11:49:11.660 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish 2022-01-15T11:54:01.489 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-15T11:54:01.517 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-15T11:54:01.792 INFO:teuthology.provision.fog.smithi022:Node is ready 2022-01-15T11:54:01.808 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com 2022-01-15T11:54:01.834 INFO:teuthology.provision.fog.smithi114:Node is ready 2022-01-15T11:54:01.849 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com 2022-01-15T11:54:01.867 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2022-01-15T11:54:01.905 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114 2022-01-15T11:54:02.197 INFO:teuthology.provision.fog.smithi022:Deploy complete! 2022-01-15T11:54:02.221 INFO:teuthology.provision.fog.smithi114:Deploy complete! 2022-01-15T11:54:02.372 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com 2022-01-15T11:54:02.373 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-15T11:54:02.392 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com 2022-01-15T11:54:02.393 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-15T11:54:02.413 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server 2022-01-15T11:54:02.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2022-01-15T11:54:02.675 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2022-01-15T11:54:02.731 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Linux" 2022-01-15T11:54:02.731 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="8 (Core)" 2022-01-15T11:54:02.731 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2022-01-15T11:54:02.731 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2022-01-15T11:54:02.732 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="8" 2022-01-15T11:54:02.732 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el8" 2022-01-15T11:54:02.733 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2022-01-15T11:54:02.733 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2022-01-15T11:54:02.733 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-15T11:54:02.733 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.centos.org/" 2022-01-15T11:54:02.734 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-15T11:54:02.734 INFO:teuthology.orchestra.run.smithi022.stdout: 2022-01-15T11:54:02.734 INFO:teuthology.orchestra.run.smithi022.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-15T11:54:02.734 INFO:teuthology.orchestra.run.smithi022.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-15T11:54:02.734 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2022-01-15T11:54:02.735 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2022-01-15T11:54:02.735 INFO:teuthology.orchestra.run.smithi022.stdout: 2022-01-15T11:54:02.736 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2022-01-15T11:54:02.761 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server 2022-01-15T11:54:02.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2022-01-15T11:54:03.014 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2022-01-15T11:54:03.069 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Linux" 2022-01-15T11:54:03.069 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="8 (Core)" 2022-01-15T11:54:03.069 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2022-01-15T11:54:03.070 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2022-01-15T11:54:03.070 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="8" 2022-01-15T11:54:03.070 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el8" 2022-01-15T11:54:03.070 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2022-01-15T11:54:03.071 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2022-01-15T11:54:03.071 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-15T11:54:03.071 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.centos.org/" 2022-01-15T11:54:03.072 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-15T11:54:03.072 INFO:teuthology.orchestra.run.smithi114.stdout: 2022-01-15T11:54:03.072 INFO:teuthology.orchestra.run.smithi114.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-15T11:54:03.072 INFO:teuthology.orchestra.run.smithi114.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-15T11:54:03.072 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2022-01-15T11:54:03.073 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2022-01-15T11:54:03.073 INFO:teuthology.orchestra.run.smithi114.stdout: 2022-01-15T11:54:03.073 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2022-01-15T11:54:03.168 INFO:teuthology.dispatcher.supervisor:Running job 6619385 2022-01-15T11:54:03.169 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3ff86a5dbaafecdb6414fa2fea970bb44e836829/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619385 --name yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} -- /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619385/orig.config.yaml 2022-01-15T11:54:03.172 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619385 2022-01-15T11:54:03.173 INFO:teuthology.dispatcher.supervisor:Job PID: 19101 2022-01-15T11:54:03.173 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-15T18:24:32.383 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-15T18:24:32.597 INFO:teuthology.kill:Killing Pids: {19101} 2022-01-15T18:24:33.709 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-15T18:24:33.710 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-15T18:24:33.710 INFO:teuthology.misc:Compressing logs... 2022-01-15T18:24:33.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2022-01-15T18:24:33.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2022-01-15T18:24:33.962 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-15T18:24:34.226 INFO:teuthology.orchestra.run.smithi114.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-15T18:24:34.285 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-15T18:24:34.285 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2022-01-15T18:24:34.316 INFO:teuthology.orchestra.run.smithi114.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-15T18:24:34.316 INFO:teuthology.orchestra.run.smithi114.stderr:tar: Error is not recoverable: exiting now 2022-01-15T18:24:34.317 INFO:teuthology.misc:Compressing logs... 2022-01-15T18:24:34.508 INFO:teuthology.misc:Compressing logs... 2022-01-15T18:24:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-15T18:24:43.698 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-15T18:24:50.223 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-15T18:24:50.769 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-15T18:24:54.238 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-mon.smithi114.log: file size changed while zipping 2022-01-15T18:25:00.064 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-mon.smithi022.log: file size changed while zipping 2022-01-15T18:25:01.747 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-15T18:25:08.787 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-15T18:25:10.690 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-mgr.smithi022.nltnre.log: file size changed while zipping 2022-01-15T18:25:10.782 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-15T18:25:21.194 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/19c6b5ce-75fb-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-15T18:25:38.648 INFO:teuthology.kill:No teuthology processes running 2022-01-15T18:25:39.768 INFO:teuthology.kill:Nuking machines: ['smithi022', 'smithi114'] 2022-01-15T18:25:40.713 INFO:teuthology.kill:2022-01-15 18:25:40,712.712 INFO:teuthology.nuke:targets: 2022-01-15T18:25:40.714 INFO:teuthology.kill: smithi022.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAFfk5WqHb3hOzFx6yduqJ/fNA2BXaydX1e1PCr9JimQtXLHWtT6Wjl1j/+nbj0JinSxA3fRRpxBysCsi4qkKGLTKM9BaLJCl+W1SkLrTeVPt2EZVNnt5HSB+GlAJ1UeK0QmeHgppPA0tLZ6ZRNQFbxIgVzYo5dvMQdYNTQnx+gmWaOUVKrBH3p7m3WWHbG+b/UoDEGcOGLA0yP65rGkiD5/91dc+FDqrmY0sWhIW9BhTaWT8CKpV6k0Zr9Ug1w1Nz7Uu6hR4IF9yyPiljh47NsMCuN3o7OXLgSIHBAV6MT1IgDi2WDGqfQqEw/9okQXpSriE0sAMc3Gg+6y9IVrvNN4U7qq6GK2Z1WggwrXotJCjpUY2bobCaZVHCkn4tu19+Gip3jEphkgOj154ILhYvFzjnhLgpOofxEy3BQBONXtcSCI7UjvUZjwy9ACzyuO2+zlMxXmX8f8imS5NM/slnmwBn3duXvYHOI4Ea1eBI9H1HEwgm9ylBKsNArOs259E= 2022-01-15T18:25:40.714 INFO:teuthology.kill: smithi114.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1Hg1aJGWTVUCUXU7vBjeiK/PmxP7s//ugepT1mVfvbzLH1DNqN6EPx2W3wYGlnypSamypGPJJBQc6lKtrGIdJoHGlTOFUQk1YCEGYBXpIqi4mO7xG0XDsXO+DOZM9GX9O6/+64AxKilID/nCWIbwNtkq7QbiOLsD/yqmxW0cHw8xGrHBO600vdwfy29SFiQolzH9/MU4HKCGdQKLEYyeOYktgOGOe4t9I+XYUanniYW4pOThntO4KqMO6cjtbJ0wHa3gohRR9Aag1kzxd3vRH3jPFrN+KAt5OgeyaxG6soqaJSdsklbahDN3KPbdpRDUJnw90IBWBX8OOI7rjTzLSSiZqxTpFql51tNrS++CTQL6aEDbSQ19UksOK8y1FviwwAaqZVoPHAVMmXVe3vDX9Lnm1+Te0nR1dLmGfmkhSajDNxLo21QU5So0Ae5YCdugwPwa1qM0oYC/o9QAjvgndm5IxBuI/QHMrG7/06uC65JK1kfC6Sb0iykkB0gAp6z8= 2022-01-15T18:25:40.739 INFO:teuthology.kill:2022-01-15 18:25:40,739.739 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-15T18:25:40.742 INFO:teuthology.kill:2022-01-15 18:25:40,742.742 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-15T18:25:40.815 INFO:teuthology.kill:2022-01-15 18:25:40,815.815 INFO:teuthology.orchestra.console:Power off smithi022 2022-01-15T18:25:40.827 INFO:teuthology.kill:2022-01-15 18:25:40,826.826 INFO:teuthology.orchestra.console:Power off smithi114 2022-01-15T18:25:49.184 INFO:teuthology.kill:2022-01-15 18:25:49,184.184 INFO:teuthology.orchestra.console:Power off for smithi022 completed 2022-01-15T18:25:49.334 INFO:teuthology.kill:2022-01-15 18:25:49,333.333 INFO:teuthology.lock.ops:unlocked: smithi022.front.sepia.ceph.com 2022-01-15T18:25:53.303 INFO:teuthology.kill:2022-01-15 18:25:53,303.303 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2022-01-15T18:25:53.447 INFO:teuthology.kill:2022-01-15 18:25:53,447.447 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com 2022-01-15T18:27:53.747 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-15T18:27:53.838 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi022 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-15_15:15:15-upgrade-wip-yuriw-quincy-1.14.22-distro-default-smithi/6619894 2022-01-15T18:27:53.839 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi114 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-15_15:15:15-upgrade-wip-yuriw-quincy-1.14.22-distro-default-smithi/6619894