2021-12-18T13:15:42.220 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-18T13:15:42.387 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi007.front.sepia.ceph.com' reimaging 2021-12-18T13:15:42.388 INFO:teuthology.lock.ops:Updating [ubuntu@smithi007.front.sepia.ceph.com]: reset os type and version on server 2021-12-18T13:15:42.389 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2021-12-18T13:15:42.409 INFO:teuthology.lock.ops:Node 'ubuntu@smithi007.front.sepia.ceph.com' reimaging is complete 2021-12-18T13:15:42.410 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi032.front.sepia.ceph.com' reimaging 2021-12-18T13:15:42.410 INFO:teuthology.lock.ops:Updating [ubuntu@smithi032.front.sepia.ceph.com]: reset os type and version on server 2021-12-18T13:15:42.410 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2021-12-18T13:15:42.430 INFO:teuthology.lock.ops:Node 'ubuntu@smithi032.front.sepia.ceph.com' reimaging is complete 2021-12-18T13:15:42.430 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi062.front.sepia.ceph.com' reimaging 2021-12-18T13:15:42.430 INFO:teuthology.lock.ops:Updating [ubuntu@smithi062.front.sepia.ceph.com]: reset os type and version on server 2021-12-18T13:15:42.431 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2021-12-18T13:15:42.448 INFO:teuthology.lock.ops:Node 'ubuntu@smithi062.front.sepia.ceph.com' reimaging is complete 2021-12-18T13:15:42.627 INFO:teuthology.provision.fog.smithi007:Scheduling deploy of centos 8.3 2021-12-18T13:15:42.658 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of centos 8.3 2021-12-18T13:15:42.661 INFO:teuthology.provision.fog.smithi032:Scheduling deploy of centos 8.3 2021-12-18T13:15:42.985 INFO:teuthology.orchestra.console:Power off smithi062 2021-12-18T13:15:43.023 INFO:teuthology.orchestra.console:Power off smithi007 2021-12-18T13:15:43.069 INFO:teuthology.orchestra.console:Power off smithi032 2021-12-18T13:15:51.383 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2021-12-18T13:15:51.401 INFO:teuthology.orchestra.console:Power off for smithi007 completed 2021-12-18T13:15:51.436 INFO:teuthology.orchestra.console:Power off for smithi032 completed 2021-12-18T13:15:51.485 INFO:teuthology.orchestra.console:Power on smithi062 2021-12-18T13:15:51.502 INFO:teuthology.orchestra.console:Power on smithi007 2021-12-18T13:15:51.537 INFO:teuthology.orchestra.console:Power on smithi032 2021-12-18T13:15:59.860 INFO:teuthology.orchestra.console:Power on for smithi062 completed 2021-12-18T13:15:59.863 INFO:teuthology.orchestra.console:Power on for smithi007 completed 2021-12-18T13:15:59.916 INFO:teuthology.orchestra.console:Power on for smithi032 completed 2021-12-18T13:15:59.961 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish 2021-12-18T13:15:59.964 INFO:teuthology.provision.fog.smithi007:Waiting for deploy to finish 2021-12-18T13:16:00.018 INFO:teuthology.provision.fog.smithi032:Waiting for deploy to finish 2021-12-18T13:19:43.917 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-18T13:19:49.466 INFO:teuthology.provision.fog.smithi032:Node is ready 2021-12-18T13:19:49.485 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032.front.sepia.ceph.com 2021-12-18T13:19:49.542 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32 smithi032.front.sepia.ceph.com smithi032 2021-12-18T13:19:49.901 INFO:teuthology.provision.fog.smithi032:Deploy complete! 2021-12-18T13:20:05.637 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-18T13:20:25.852 INFO:teuthology.provision.fog.smithi062:Node is ready 2021-12-18T13:20:25.871 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com 2021-12-18T13:20:25.929 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062 2021-12-18T13:20:26.288 INFO:teuthology.provision.fog.smithi062:Deploy complete! 2021-12-18T13:20:49.235 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-18T13:20:49.557 INFO:teuthology.provision.fog.smithi007:Node is ready 2021-12-18T13:20:49.572 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007.front.sepia.ceph.com 2021-12-18T13:20:49.628 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7 smithi007.front.sepia.ceph.com smithi007 2021-12-18T13:20:49.946 INFO:teuthology.provision.fog.smithi007:Deploy complete! 2021-12-18T13:20:50.146 INFO:teuthology.lock.ops:Checking smithi007.front.sepia.ceph.com 2021-12-18T13:20:50.147 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-18T13:20:50.175 INFO:teuthology.lock.ops:Checking smithi032.front.sepia.ceph.com 2021-12-18T13:20:50.176 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-18T13:20:50.192 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com 2021-12-18T13:20:50.192 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-18T13:20:50.212 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: set os type and version on server 2021-12-18T13:20:50.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2021-12-18T13:20:50.413 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2021-12-18T13:20:50.468 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Linux" 2021-12-18T13:20:50.468 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8" 2021-12-18T13:20:50.468 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2021-12-18T13:20:50.468 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2021-12-18T13:20:50.468 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8" 2021-12-18T13:20:50.469 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2021-12-18T13:20:50.469 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-18T13:20:50.469 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2021-12-18T13:20:50.469 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-18T13:20:50.470 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2021-12-18T13:20:50.470 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-18T13:20:50.470 INFO:teuthology.orchestra.run.smithi007.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-18T13:20:50.470 INFO:teuthology.orchestra.run.smithi007.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-18T13:20:50.471 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2021-12-18T13:20:50.489 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: set os type and version on server 2021-12-18T13:20:50.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2021-12-18T13:20:50.699 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2021-12-18T13:20:50.756 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Linux" 2021-12-18T13:20:50.757 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8" 2021-12-18T13:20:50.757 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2021-12-18T13:20:50.757 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2021-12-18T13:20:50.757 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8" 2021-12-18T13:20:50.757 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2021-12-18T13:20:50.758 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-18T13:20:50.758 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2021-12-18T13:20:50.758 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-18T13:20:50.758 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2021-12-18T13:20:50.759 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-18T13:20:50.759 INFO:teuthology.orchestra.run.smithi032.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-18T13:20:50.759 INFO:teuthology.orchestra.run.smithi032.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-18T13:20:50.759 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2021-12-18T13:20:50.776 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server 2021-12-18T13:20:50.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2021-12-18T13:20:50.985 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2021-12-18T13:20:51.041 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Linux" 2021-12-18T13:20:51.042 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8" 2021-12-18T13:20:51.042 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2021-12-18T13:20:51.042 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2021-12-18T13:20:51.042 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8" 2021-12-18T13:20:51.043 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2021-12-18T13:20:51.043 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-18T13:20:51.043 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2021-12-18T13:20:51.043 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-18T13:20:51.043 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2021-12-18T13:20:51.044 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-18T13:20:51.044 INFO:teuthology.orchestra.run.smithi062.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-18T13:20:51.044 INFO:teuthology.orchestra.run.smithi062.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-18T13:20:51.044 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2021-12-18T13:20:51.140 INFO:teuthology.dispatcher.supervisor:Running job 6569693 2021-12-18T13:20:51.141 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-12-18_13:15:02-multimds-octopus-distro-default-smithi/6569693 --name teuthology-2021-12-18_13:15:02-multimds-octopus-distro-default-smithi --description multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} -- /home/teuthworker/archive/teuthology-2021-12-18_13:15:02-multimds-octopus-distro-default-smithi/6569693/orig.config.yaml 2021-12-18T13:20:51.144 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-12-18_13:15:02-multimds-octopus-distro-default-smithi/6569693 2021-12-18T13:20:51.145 INFO:teuthology.dispatcher.supervisor:Job PID: 12793 2021-12-18T13:20:51.145 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-19T01:21:11.305 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-19T01:21:11.504 INFO:teuthology.kill:Killing Pids: {12793} 2021-12-19T01:21:12.889 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2021-12-19T01:21:12.890 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'mds.b', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-12-19T01:21:12.891 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['client.0', 'client.1'] 2021-12-19T01:21:12.891 INFO:teuthology.misc:Compressing logs... 2021-12-19T01:21:12.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2021-12-19T01:21:13.145 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2021-12-19T01:21:13.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2021-12-19T01:21:13.830 INFO:teuthology.kill:No teuthology processes running 2021-12-19T01:21:15.080 INFO:teuthology.kill:Nuking machines: ['smithi007', 'smithi032', 'smithi062'] 2021-12-19T01:21:16.185 INFO:teuthology.kill:2021-12-19 01:21:16,185.185 INFO:teuthology.nuke:targets: 2021-12-19T01:21:16.186 INFO:teuthology.kill: smithi007.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTwv3fs86VcZPX0THPAK2DbQxT3flgEQpfQmbL/7bax+pJouJQvuP/QSBGqo64HtGuRDrxtMbp71C59gScuM785U17/Kl9pEPm3oSaXRJz3iHuvNkXWmwHTKs2f7o9xULPIXmB05XYrjfy7ra3GNfN0DRNdN06Bxw6cltGKUzOcgha6YB9f4E4DiLSqO++l7L/jBVjTksmUOnFqnnuv9uM0AocKi0EVkY3iE52g/tfm8sPCNqahEYYT3uo+vuPSFhXpU0AdhWA5wqBo6GQPh+o3+JLG4PIjeqmlHZPYxd6TPYL2bRfq9ywxa/2R0QP2wQEEcEMekdOveepcSQuRLBAHhxguky/g6pSkojjGMqg+30X6Pck17/IeGAU+afiIENp9T1XqaWmTZ55lUpWnM/xehr+3Uyd/ZjQ93jgD7itd5H3LMV7kmQFodR7pSAIJJP8H3kHKE9Uf70wRc7yVAb6GQPKKwLHjrSl+3O+6nQFoC5HUgNS3Qy5NkkT4lMIZGM= 2021-12-19T01:21:16.187 INFO:teuthology.kill: smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuPQkJYHqwPp0hroTqUHBYV6l0Uh/zz/C/WVP/jBN1vjPiS0Ml/DPWFoWv9hskAAcqi+t4s463Teq9F8F9Ij3xCagIrJoL9nN7UbGElF7sFaO2tTLEyq0ubFVYvMt5wWFITK2gO3LdNgPTyBr+T6GP9Kt65VQwPlEoEnHUeJ2IcV+W/ez8NI+akvIO7/tzTiIchT2v6piQhThauScuKpYFEjVMvvESjoS7sUMqE/rZz7dhSqaPl/Fi+2+75FGBB3tTfeywIsVpyCKWzvh1OjBxvbvm5LF+Qz+cKpbD4YV/yN2vG6didUDgObUPdR7xbnEUofbs3KD/6zEwRt9nKgVJRRvfVub28bMFhPh5/Ww6XDPYnIwf2HxqDKcPURjAXcnII5sLT8BWLz6DMp3dX8f9LkW9pbIjc2u8JQYH+X8FQf5gOknDr3C1xl0QmwZf4eWuLqy4oXDvjwVJzAtEclpY/nEwpGh0d6qOS1sghJIpsdFOKOv5RejJiNYPC9laHUk= 2021-12-19T01:21:16.187 INFO:teuthology.kill: smithi062.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgx88fiiDqa8XCTSTMED0iHSl3WXJ1FZlmIisplpCM9PhQ+TYMBZfqDdNShkYuzlZ01VShGhpKP0EFQWZZCREU+xw3C5fQepxW+sjza7QjCpcjMCuv5x509w7gy7/ciO0FdqOIKy711SWUsggORg5tB/uKw10WLHT6dw2ifKVF1WkXDEWyE+1mbB1eVK7top+e64K1Xtb+VL32quKFyRwuYZlHmrZOP4ccGePceuKAHELhVf77PjAoC55p4sx6XUcV9lBEq7jbs6oYw1efmEsjkyWoESRM3SsgtSIS8M8jeWgHYqG6afqquTaJwi7dZ80z4VExZRhMMwKSfLCEtHC6C4XChKRs5M8s/W93GWkPhaYj23GZnEv0stsVrBP5cmcPdXJMDkakXWJQQ5jHzjKwcGcDH1iKo7rd6SKxAcADS2BAit/YF9ESooZ7rSVbVrbgJI/8M/vI+6vKhI1ut4PfjxgnO0Tr959SbXoP4+UdYlYOs6Zw1Mvmbai1VdRas3E= 2021-12-19T01:21:16.216 INFO:teuthology.kill:2021-12-19 01:21:16,215.215 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-19T01:21:16.221 INFO:teuthology.kill:2021-12-19 01:21:16,220.220 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-19T01:21:16.222 INFO:teuthology.kill:2021-12-19 01:21:16,221.221 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-19T01:21:16.308 INFO:teuthology.kill:2021-12-19 01:21:16,308.308 INFO:teuthology.orchestra.console:Power off smithi032 2021-12-19T01:21:16.324 INFO:teuthology.kill:2021-12-19 01:21:16,324.324 INFO:teuthology.orchestra.console:Power off smithi007 2021-12-19T01:21:16.336 INFO:teuthology.kill:2021-12-19 01:21:16,335.335 INFO:teuthology.orchestra.console:Power off smithi062 2021-12-19T01:21:24.699 INFO:teuthology.kill:2021-12-19 01:21:24,698.698 INFO:teuthology.orchestra.console:Power off for smithi032 completed 2021-12-19T01:21:24.705 INFO:teuthology.kill:2021-12-19 01:21:24,705.705 INFO:teuthology.orchestra.console:Power off for smithi007 completed 2021-12-19T01:21:24.712 INFO:teuthology.kill:2021-12-19 01:21:24,711.711 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2021-12-19T01:21:24.868 INFO:teuthology.kill:2021-12-19 01:21:24,866.866 INFO:teuthology.lock.ops:unlocked: smithi032.front.sepia.ceph.com 2021-12-19T01:21:24.870 INFO:teuthology.kill:2021-12-19 01:21:24,870.870 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com 2021-12-19T01:21:24.874 INFO:teuthology.kill:2021-12-19 01:21:24,874.874 INFO:teuthology.lock.ops:unlocked: smithi007.front.sepia.ceph.com 2021-12-19T01:23:25.254 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-19T01:23:25.389 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi007 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-18_18:14:24-rados-wip-yuriw-master-12.18.21-distro-default-smithi/6570212 2021-12-19T01:23:25.389 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi032 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-18_18:14:24-rados-wip-yuriw-master-12.18.21-distro-default-smithi/6570216 2021-12-19T01:23:25.390 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi062 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-18_18:14:24-rados-wip-yuriw-master-12.18.21-distro-default-smithi/6570212