2022-01-08T23:02:06.863 INFO:root:teuthology version: 1.1.0-95a7d479 2022-01-08T23:02:06.998 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi066.front.sepia.ceph.com' reimaging 2022-01-08T23:02:06.999 INFO:teuthology.lock.ops:Updating [ubuntu@smithi066.front.sepia.ceph.com]: reset os type and version on server 2022-01-08T23:02:06.999 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2022-01-08T23:02:07.019 INFO:teuthology.lock.ops:Node 'ubuntu@smithi066.front.sepia.ceph.com' reimaging is complete 2022-01-08T23:02:07.020 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi151.front.sepia.ceph.com' reimaging 2022-01-08T23:02:07.021 INFO:teuthology.lock.ops:Updating [ubuntu@smithi151.front.sepia.ceph.com]: reset os type and version on server 2022-01-08T23:02:07.021 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2022-01-08T23:02:07.045 INFO:teuthology.lock.ops:Node 'ubuntu@smithi151.front.sepia.ceph.com' reimaging is complete 2022-01-08T23:02:07.229 INFO:teuthology.provision.fog.smithi066:Scheduling deploy of centos 8.3 2022-01-08T23:02:07.250 INFO:teuthology.provision.fog.smithi151:Scheduling deploy of centos 8.3 2022-01-08T23:02:07.580 INFO:teuthology.orchestra.console:Power off smithi066 2022-01-08T23:02:07.758 INFO:teuthology.orchestra.console:Power off smithi151 2022-01-08T23:02:15.956 INFO:teuthology.orchestra.console:Power off for smithi066 completed 2022-01-08T23:02:16.058 INFO:teuthology.orchestra.console:Power on smithi066 2022-01-08T23:02:20.239 INFO:teuthology.orchestra.console:Power off for smithi151 completed 2022-01-08T23:02:20.340 INFO:teuthology.orchestra.console:Power on smithi151 2022-01-08T23:02:24.434 INFO:teuthology.orchestra.console:Power on for smithi066 completed 2022-01-08T23:02:24.536 INFO:teuthology.provision.fog.smithi066:Waiting for deploy to finish 2022-01-08T23:02:28.707 INFO:teuthology.orchestra.console:Power on for smithi151 completed 2022-01-08T23:02:28.809 INFO:teuthology.provision.fog.smithi151:Waiting for deploy to finish 2022-01-08T23:07:33.725 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-08T23:07:34.007 INFO:teuthology.provision.fog.smithi151:Node is ready 2022-01-08T23:07:34.022 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151.front.sepia.ceph.com 2022-01-08T23:07:34.077 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151 smithi151.front.sepia.ceph.com smithi151 2022-01-08T23:07:34.399 INFO:teuthology.provision.fog.smithi151:Deploy complete! 2022-01-08T23:07:44.438 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-08T23:07:44.724 INFO:teuthology.provision.fog.smithi066:Node is ready 2022-01-08T23:07:44.780 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066.front.sepia.ceph.com 2022-01-08T23:07:44.836 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66 smithi066.front.sepia.ceph.com smithi066 2022-01-08T23:07:45.158 INFO:teuthology.provision.fog.smithi066:Deploy complete! 2022-01-08T23:07:45.315 INFO:teuthology.lock.ops:Checking smithi066.front.sepia.ceph.com 2022-01-08T23:07:45.315 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-08T23:07:45.334 INFO:teuthology.lock.ops:Checking smithi151.front.sepia.ceph.com 2022-01-08T23:07:45.335 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-08T23:07:45.353 INFO:teuthology.lock.ops:Updating [smithi066.front.sepia.ceph.com]: set os type and version on server 2022-01-08T23:07:45.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2022-01-08T23:07:45.552 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2022-01-08T23:07:45.607 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Linux" 2022-01-08T23:07:45.608 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8" 2022-01-08T23:07:45.608 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2022-01-08T23:07:45.608 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2022-01-08T23:07:45.609 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8" 2022-01-08T23:07:45.609 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8" 2022-01-08T23:07:45.609 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-08T23:07:45.609 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2022-01-08T23:07:45.609 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-08T23:07:45.610 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2022-01-08T23:07:45.610 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-08T23:07:45.610 INFO:teuthology.orchestra.run.smithi066.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-08T23:07:45.611 INFO:teuthology.orchestra.run.smithi066.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-08T23:07:45.612 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2022-01-08T23:07:45.630 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: set os type and version on server 2022-01-08T23:07:45.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2022-01-08T23:07:45.899 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2022-01-08T23:07:45.953 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Linux" 2022-01-08T23:07:45.953 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8" 2022-01-08T23:07:45.954 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2022-01-08T23:07:45.954 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2022-01-08T23:07:45.954 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8" 2022-01-08T23:07:45.954 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8" 2022-01-08T23:07:45.954 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-08T23:07:45.955 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2022-01-08T23:07:45.955 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-08T23:07:45.955 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2022-01-08T23:07:45.955 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-08T23:07:45.956 INFO:teuthology.orchestra.run.smithi151.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-08T23:07:45.956 INFO:teuthology.orchestra.run.smithi151.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-08T23:07:45.956 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2022-01-08T23:07:46.059 INFO:teuthology.dispatcher.supervisor:Running job 6602658 2022-01-08T23:07:46.059 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-08_15:53:33-rados-wip-yuriw-master-1.7.22-distro-default-smithi/6602658 --name yuriw-2022-01-08_15:53:33-rados-wip-yuriw-master-1.7.22-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/yuriw-2022-01-08_15:53:33-rados-wip-yuriw-master-1.7.22-distro-default-smithi/6602658/orig.config.yaml 2022-01-08T23:07:46.063 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-08_15:53:33-rados-wip-yuriw-master-1.7.22-distro-default-smithi/6602658 2022-01-08T23:07:46.064 INFO:teuthology.dispatcher.supervisor:Job PID: 2215 2022-01-08T23:07:46.064 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-09T11:08:39.610 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-09T11:08:39.804 INFO:teuthology.kill:Killing Pids: {2215} 2022-01-09T11:08:40.941 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-09T11:08:40.942 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-09T11:08:40.943 INFO:teuthology.misc:Compressing logs... 2022-01-09T11:08:40.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2022-01-09T11:08:41.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2022-01-09T11:08:41.196 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-09T11:08:41.398 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-09T11:08:41.442 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-09T11:08:41.443 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2022-01-09T11:08:41.475 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-09T11:08:41.475 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2022-01-09T11:08:41.476 INFO:teuthology.misc:Compressing logs... 2022-01-09T11:08:41.694 INFO:teuthology.misc:Compressing logs... 2022-01-09T11:08:55.475 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-09T11:08:58.089 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-09T11:09:12.619 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-09T11:09:13.244 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-09T11:09:19.740 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-mon.smithi151.log: file size changed while zipping 2022-01-09T11:09:32.151 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-09T11:09:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-09T11:09:50.318 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-09T11:09:50.666 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-mon.smithi066.log: file size changed while zipping 2022-01-09T11:10:05.266 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-09T11:10:08.748 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/2002d692-70d9-11ec-8c32-001a4aab830c/ceph-mgr.smithi066.nmrnra.log: file size changed while zipping 2022-01-09T11:10:40.844 INFO:teuthology.kill:No teuthology processes running 2022-01-09T11:10:41.992 INFO:teuthology.kill:Nuking machines: ['smithi066', 'smithi151'] 2022-01-09T11:10:42.947 INFO:teuthology.kill:2022-01-09 11:10:42,947.947 INFO:teuthology.nuke:targets: 2022-01-09T11:10:42.948 INFO:teuthology.kill: smithi066.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD06BwI8loBE6LLfGfPwoCZPGxBSuO0i+cbLqFxPjEBIPJ5Bz8r8dDLJ9sh5H1yZ9FUvsz/DZ3J1XWmgvxNKhyL9MwJ4V/eVSkbaE+jyqSIPxu9nxSk+rlrCrnxe05RgyXdqYakq2wnhxuhc6tx2l4AtDEIDRBAlxZhuYRMaVB8yqU7PzDuw8FKbud4nO4iYGX9+kOUEA+LSwBIZD/nHdOjO3H9tTZ2nMZ7kkibV6aKOCTFsBmGWdPYrkd8yELiCD9HjtJ7uRYBnrWVFsXvpU4a5cHcCQUnCp6xQDvSJJLmN/JuMNk4ANv3flZ9sxnsKiBQeaQXk5mAkebPY3GakOCsOUWdjWDaqfs37dUTRW3I8dZxtKN4IUJthuzHpIr0TUotAdB2ju1me5T1EkULRMXPLc46MABE5puj1jLxxERXOjGAXk1vQMqNMMt1MBbgClgFhW3AonC2uvmVqKcN3zcFQbXZ04xVNjZYFI7vi+VGADgFuyMaygZea7bx7VW/fgc= 2022-01-09T11:10:42.949 INFO:teuthology.kill: smithi151.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+TbQrSRyI25bKKqhgfnDaIduRyKVmsT4jhXaIpwFJ/7lLj/PJu1cxUt6TQ0nT60yU8CeRThdKVJzpc6zWCh5XYJNBGW8wz1gtHriCbBIoAUd/YXrtvbfBb5Hi6cYXbqq3tqC69lnTvGdK7ZlEI60LDJlFoHk9e66VNERl5I0ZrlJfMP2FLg1+qOJ+D6D7Gp+75f2IEzJp5mrmFIlLg9s0O3js0iyGmnXewOzCwEhdn+FF2UJfGgGY/69fDIgCPJcf/I6MrVzJ0f61n+nVrl44CcLU3x4tuZ2AFESngQe38Lya0VkYDFFleJvxfTxut8/1hnMzMtPMKUKtmvARc305v0Vv2/xTDPyO4FieMTlTLZt4/NtPaBrUo+8C0FLlfsPdPx9pO0iinmFeuEmwgi/sB3q5WysKac0oNDfOs1HVtQRrLVjQerDHw5+CMW5+fg5rbKeI5bG6GRGtvNa0eLTh2nJf8XWwEX3uEr1EmuKVaFbR9Kx35kIrZI6BmT5+Lzk= 2022-01-09T11:10:42.974 INFO:teuthology.kill:2022-01-09 11:10:42,974.974 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-09T11:10:42.978 INFO:teuthology.kill:2022-01-09 11:10:42,977.977 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-09T11:10:43.050 INFO:teuthology.kill:2022-01-09 11:10:43,050.050 INFO:teuthology.orchestra.console:Power off smithi066 2022-01-09T11:10:43.064 INFO:teuthology.kill:2022-01-09 11:10:43,063.063 INFO:teuthology.orchestra.console:Power off smithi151 2022-01-09T11:10:51.429 INFO:teuthology.kill:2022-01-09 11:10:51,429.429 INFO:teuthology.orchestra.console:Power off for smithi066 completed 2022-01-09T11:10:51.434 INFO:teuthology.kill:2022-01-09 11:10:51,434.434 INFO:teuthology.orchestra.console:Power off for smithi151 completed 2022-01-09T11:10:51.575 INFO:teuthology.kill:2022-01-09 11:10:51,574.574 INFO:teuthology.lock.ops:unlocked: smithi066.front.sepia.ceph.com 2022-01-09T11:10:51.578 INFO:teuthology.kill:2022-01-09 11:10:51,578.578 INFO:teuthology.lock.ops:unlocked: smithi151.front.sepia.ceph.com 2022-01-09T11:12:51.809 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-09T11:12:51.912 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi066 but it was locked by another job: /home/teuthworker/archive/oro-2022-01-09_11:03:08-rbd-wip-oro-rbd-external-migrate-distro-default-smithi/6604703 2022-01-09T11:12:51.912 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi151 but it was locked by another job: /home/teuthworker/archive/oro-2022-01-09_11:03:08-rbd-wip-oro-rbd-external-migrate-distro-default-smithi/6604703