2022-01-13T14:07:08.204 INFO:root:teuthology version: 1.1.0-00e09f05 2022-01-13T14:07:08.329 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging 2022-01-13T14:07:08.330 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server 2022-01-13T14:07:08.331 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2022-01-13T14:07:08.349 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete 2022-01-13T14:07:08.349 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging 2022-01-13T14:07:08.350 INFO:teuthology.lock.ops:Updating [ubuntu@smithi112.front.sepia.ceph.com]: reset os type and version on server 2022-01-13T14:07:08.350 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2022-01-13T14:07:08.368 INFO:teuthology.lock.ops:Node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging is complete 2022-01-13T14:07:08.557 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 8.3 2022-01-13T14:07:08.583 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 8.3 2022-01-13T14:07:08.926 INFO:teuthology.orchestra.console:Power off smithi110 2022-01-13T14:07:08.962 INFO:teuthology.orchestra.console:Power off smithi112 2022-01-13T14:07:09.071 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2022-01-13T14:07:09.104 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2022-01-13T14:07:09.174 INFO:teuthology.orchestra.console:Power on smithi110 2022-01-13T14:07:09.205 INFO:teuthology.orchestra.console:Power on smithi112 2022-01-13T14:07:13.435 INFO:teuthology.orchestra.console:Power on for smithi110 completed 2022-01-13T14:07:13.457 INFO:teuthology.orchestra.console:Power on for smithi112 completed 2022-01-13T14:07:13.537 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish 2022-01-13T14:07:13.558 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2022-01-13T14:12:03.227 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-13T14:12:03.518 INFO:teuthology.provision.fog.smithi112:Node is ready 2022-01-13T14:12:03.533 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2022-01-13T14:12:03.589 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2022-01-13T14:12:03.914 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2022-01-13T14:12:18.265 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-13T14:12:18.571 INFO:teuthology.provision.fog.smithi110:Node is ready 2022-01-13T14:12:18.586 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com 2022-01-13T14:12:18.641 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110 2022-01-13T14:12:18.962 INFO:teuthology.provision.fog.smithi110:Deploy complete! 2022-01-13T14:12:19.109 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com 2022-01-13T14:12:19.110 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-13T14:12:19.128 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2022-01-13T14:12:19.129 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-13T14:12:19.147 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server 2022-01-13T14:12:19.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2022-01-13T14:12:19.348 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2022-01-13T14:12:19.403 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Linux" 2022-01-13T14:12:19.403 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8" 2022-01-13T14:12:19.403 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2022-01-13T14:12:19.403 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2022-01-13T14:12:19.404 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8" 2022-01-13T14:12:19.404 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8" 2022-01-13T14:12:19.404 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-13T14:12:19.404 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2022-01-13T14:12:19.405 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-13T14:12:19.405 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2022-01-13T14:12:19.405 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-13T14:12:19.405 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-13T14:12:19.406 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-13T14:12:19.407 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2022-01-13T14:12:19.426 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2022-01-13T14:12:19.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2022-01-13T14:12:19.697 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2022-01-13T14:12:19.751 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Linux" 2022-01-13T14:12:19.752 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8" 2022-01-13T14:12:19.752 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2022-01-13T14:12:19.752 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2022-01-13T14:12:19.752 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8" 2022-01-13T14:12:19.753 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2022-01-13T14:12:19.753 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-13T14:12:19.753 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2022-01-13T14:12:19.753 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-13T14:12:19.754 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2022-01-13T14:12:19.754 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-13T14:12:19.754 INFO:teuthology.orchestra.run.smithi112.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-13T14:12:19.755 INFO:teuthology.orchestra.run.smithi112.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-13T14:12:19.755 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2022-01-13T14:12:19.848 INFO:teuthology.dispatcher.supervisor:Running job 6610231 2022-01-13T14:12:19.848 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_00e09f05e9fb2ab7c7f497f5e446613531743ae2/virtualenv/bin/teuthology -v --owner scheduled_amathuri@teuthology --archive /home/teuthworker/archive/amathuri-2022-01-12_15:07:39-rados-wip-amathuria-scrub-stats-distro-basic-smithi/6610231 --name amathuri-2022-01-12_15:07:39-rados-wip-amathuria-scrub-stats-distro-basic-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/amathuri-2022-01-12_15:07:39-rados-wip-amathuria-scrub-stats-distro-basic-smithi/6610231/orig.config.yaml 2022-01-13T14:12:19.851 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/amathuri-2022-01-12_15:07:39-rados-wip-amathuria-scrub-stats-distro-basic-smithi/6610231 2022-01-13T14:12:19.852 INFO:teuthology.dispatcher.supervisor:Job PID: 11088 2022-01-13T14:12:19.852 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-14T02:13:13.921 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-14T02:13:14.148 INFO:teuthology.kill:Killing Pids: {11088} 2022-01-14T02:13:16.184 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-14T02:13:16.188 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-14T02:13:16.188 INFO:teuthology.misc:Compressing logs... 2022-01-14T02:13:16.189 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2022-01-14T02:13:16.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2022-01-14T02:13:16.441 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-14T02:13:16.705 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-14T02:13:16.738 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-14T02:13:16.739 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2022-01-14T02:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-14T02:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2022-01-14T02:13:16.772 INFO:teuthology.misc:Compressing logs... 2022-01-14T02:13:16.990 INFO:teuthology.misc:Compressing logs... 2022-01-14T02:13:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-14T02:13:31.671 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-14T02:13:37.771 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-mon.smithi112.log: file size changed while zipping 2022-01-14T02:13:44.833 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-14T02:13:55.752 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-14T02:14:01.505 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-mon.smithi110.log: file size changed while zipping 2022-01-14T02:14:04.652 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-mgr.smithi110.ljqwcy.log: file size changed while zipping 2022-01-14T02:14:09.119 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-14T02:14:22.101 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-14T02:14:27.057 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-14T02:14:35.286 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /var/log/ceph/0dda27f6-747e-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-14T02:15:06.191 INFO:teuthology.kill:No teuthology processes running 2022-01-14T02:15:07.260 INFO:teuthology.kill:Nuking machines: ['smithi110', 'smithi112'] 2022-01-14T02:15:08.202 INFO:teuthology.kill:2022-01-14 02:15:08,201.201 INFO:teuthology.nuke:targets: 2022-01-14T02:15:08.203 INFO:teuthology.kill: smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5POrbp2BucDftS4F6Gv+tTaeJk7JGS8i3HoEMFJrxzwETdnKIY60Wx2JlUhm6MJIHoptup89Vwc6AOxTcbCc7ZcJcoFSphDNlrsJqpe6xgwXwjjxPrEnw29IMbCqNn8C/XilUQ6LRL+u9srePJnoqiMj5textAIZbfhuquluVRJ7n8ef2614SL52o/qVr3sz4oRvyVdIDbUv9a13LshxGWGBxHejUyoYkcpoEi7VbB1EOoDEP9NxF0izCOYdf6wQMC0OBKniFGQViMJGIb0qyVZwHpp59WOGX64eHtDBwyHYAtLIiQOAEqY19lKHUtJ4G3qRsk+yORfE7foP9q8TrFenqa0vtekZldapH0RrTVbYdDRSJYrWyYVOY8X2Loiqan4+Q7MtYUHXdVNcPIxAj9l6VpWoB/4vzBCC2H89WvqM7eD21teYRPF1p2BZYKQnqfIHotStW0OHnARH+nzyUlCAALTCQqDVBPiXw4jpMwvezoq7/reD5eDhpM3T+BsM= 2022-01-14T02:15:08.203 INFO:teuthology.kill: smithi112.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsQ9kdJGwnrrRqwN1gstAmTHK81KRFHZom+Z2quxHn1wDa0rJgJSUgdqHE5wXA6XLjFcCzjkaUHJmiBdirUgdtUY2NAOZckPxoSKTPZCrDWdEX7HEFjkTEQthiwdwpB0nmPZ9plDcdvRwKwN/QDTqoWyc9FDRWDqUyb0ggnGDEPAiQQUqrbBul6FDAe/NVqGBgTcg7m4PG+pGiO0APabyRcnyDhHC1osZPwFK+MeVnyumfQjdUw8I5j+64gJbhvk2hppI5PMaUiRaYthNG5+M0gTi+7oBmkZGSNPzgj8avP7t2LDKZnFGIQLUXrp4jruC0m7WRExHzQMf9N/AH5Xp6wQ7SCm0aZDHNtuCXm+xQsz4LC1M3gfEUcmuuNwfi4zWep2UTD20Zym7herljT4+6dXsHaralMasCjk9nP0hOfNVy6vkoQRoXa/torYWO6IFst5xXIockxYXQ4f9jaqTk1JwzIK48kkcS+DqrYhjIyQcp7gesteblVJy7YK7xu0s= 2022-01-14T02:15:08.228 INFO:teuthology.kill:2022-01-14 02:15:08,228.228 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-14T02:15:08.232 INFO:teuthology.kill:2022-01-14 02:15:08,232.232 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-14T02:15:08.301 INFO:teuthology.kill:2022-01-14 02:15:08,301.301 INFO:teuthology.orchestra.console:Power off smithi110 2022-01-14T02:15:08.341 INFO:teuthology.kill:2022-01-14 02:15:08,341.341 INFO:teuthology.orchestra.console:Power off smithi112 2022-01-14T02:15:16.673 INFO:teuthology.kill:2022-01-14 02:15:16,673.673 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2022-01-14T02:15:16.704 INFO:teuthology.kill:2022-01-14 02:15:16,703.703 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2022-01-14T02:15:16.818 INFO:teuthology.kill:2022-01-14 02:15:16,817.817 INFO:teuthology.lock.ops:unlocked: smithi110.front.sepia.ceph.com 2022-01-14T02:15:16.846 INFO:teuthology.kill:2022-01-14 02:15:16,846.846 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2022-01-14T02:17:17.151 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-14T02:17:17.246 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi110 but it was locked by another job: /home/teuthworker/archive/dis-2022-01-13_22:23:59-rbd-wip-dis-testing-distro-default-smithi/6614985 2022-01-14T02:17:17.247 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi112 but it was locked by another job: /home/teuthworker/archive/dis-2022-01-13_22:23:59-rbd-wip-dis-testing-distro-default-smithi/6614985