2021-12-08T10:44:04.790 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-08T10:44:04.940 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi148.front.sepia.ceph.com' reimaging 2021-12-08T10:44:04.941 INFO:teuthology.lock.ops:Updating [ubuntu@smithi148.front.sepia.ceph.com]: reset os type and version on server 2021-12-08T10:44:04.942 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2021-12-08T10:44:04.964 INFO:teuthology.lock.ops:Node 'ubuntu@smithi148.front.sepia.ceph.com' reimaging is complete 2021-12-08T10:44:04.965 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging 2021-12-08T10:44:04.965 INFO:teuthology.lock.ops:Updating [ubuntu@smithi150.front.sepia.ceph.com]: reset os type and version on server 2021-12-08T10:44:04.965 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2021-12-08T10:44:04.983 INFO:teuthology.lock.ops:Node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging is complete 2021-12-08T10:44:04.983 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi180.front.sepia.ceph.com' reimaging 2021-12-08T10:44:04.984 INFO:teuthology.lock.ops:Updating [ubuntu@smithi180.front.sepia.ceph.com]: reset os type and version on server 2021-12-08T10:44:04.984 INFO:teuthology.lock.ops:Updating smithi180.front.sepia.ceph.com on lock server 2021-12-08T10:44:05.001 INFO:teuthology.lock.ops:Node 'ubuntu@smithi180.front.sepia.ceph.com' reimaging is complete 2021-12-08T10:44:05.206 INFO:teuthology.provision.fog.smithi180:Scheduling deploy of centos 8.3 2021-12-08T10:44:05.226 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of centos 8.3 2021-12-08T10:44:05.230 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of centos 8.3 2021-12-08T10:44:05.594 INFO:teuthology.orchestra.console:Power off smithi180 2021-12-08T10:44:05.635 INFO:teuthology.orchestra.console:Power off smithi148 2021-12-08T10:44:05.692 INFO:teuthology.orchestra.console:Power off smithi150 2021-12-08T10:44:13.969 INFO:teuthology.orchestra.console:Power off for smithi180 completed 2021-12-08T10:44:14.057 INFO:teuthology.orchestra.console:Power off for smithi150 completed 2021-12-08T10:44:14.071 INFO:teuthology.orchestra.console:Power on smithi180 2021-12-08T10:44:14.158 INFO:teuthology.orchestra.console:Power on smithi150 2021-12-08T10:44:18.119 INFO:teuthology.orchestra.console:Power off for smithi148 completed 2021-12-08T10:44:18.221 INFO:teuthology.orchestra.console:Power on smithi148 2021-12-08T10:44:22.517 INFO:teuthology.orchestra.console:Power on for smithi150 completed 2021-12-08T10:44:22.529 INFO:teuthology.orchestra.console:Power on for smithi180 completed 2021-12-08T10:44:22.619 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish 2021-12-08T10:44:22.631 INFO:teuthology.provision.fog.smithi180:Waiting for deploy to finish 2021-12-08T10:44:26.584 INFO:teuthology.orchestra.console:Power on for smithi148 completed 2021-12-08T10:44:26.685 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish 2021-12-08T10:48:38.611 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-08T10:48:59.102 INFO:teuthology.provision.fog.smithi148:Node is ready 2021-12-08T10:48:59.121 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148.front.sepia.ceph.com 2021-12-08T10:48:59.178 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148 2021-12-08T10:48:59.535 INFO:teuthology.provision.fog.smithi148:Deploy complete! 2021-12-08T10:49:12.380 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-08T10:49:12.686 INFO:teuthology.provision.fog.smithi180:Node is ready 2021-12-08T10:49:12.701 INFO:teuthology.orchestra.run.smithi180.stdout:smithi180.front.sepia.ceph.com 2021-12-08T10:49:12.755 INFO:teuthology.orchestra.run.smithi180.stdout:172.21.15.180 smithi180.front.sepia.ceph.com smithi180 2021-12-08T10:49:13.066 INFO:teuthology.provision.fog.smithi180:Deploy complete! 2021-12-08T10:49:27.348 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-08T10:49:27.669 INFO:teuthology.provision.fog.smithi150:Node is ready 2021-12-08T10:49:27.684 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com 2021-12-08T10:49:27.742 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150 2021-12-08T10:49:28.071 INFO:teuthology.provision.fog.smithi150:Deploy complete! 2021-12-08T10:49:28.258 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com 2021-12-08T10:49:28.259 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-08T10:49:28.285 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com 2021-12-08T10:49:28.285 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-08T10:49:28.305 INFO:teuthology.lock.ops:Checking smithi180.front.sepia.ceph.com 2021-12-08T10:49:28.306 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-08T10:49:28.323 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server 2021-12-08T10:49:28.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2021-12-08T10:49:28.595 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2021-12-08T10:49:28.649 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Linux" 2021-12-08T10:49:28.650 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8" 2021-12-08T10:49:28.650 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2021-12-08T10:49:28.650 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2021-12-08T10:49:28.651 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8" 2021-12-08T10:49:28.651 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8" 2021-12-08T10:49:28.651 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-08T10:49:28.651 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2021-12-08T10:49:28.651 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-08T10:49:28.652 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2021-12-08T10:49:28.652 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-08T10:49:28.652 INFO:teuthology.orchestra.run.smithi148.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-08T10:49:28.652 INFO:teuthology.orchestra.run.smithi148.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-08T10:49:28.653 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2021-12-08T10:49:28.670 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server 2021-12-08T10:49:28.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2021-12-08T10:49:28.876 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2021-12-08T10:49:28.929 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Linux" 2021-12-08T10:49:28.930 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8" 2021-12-08T10:49:28.930 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2021-12-08T10:49:28.930 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2021-12-08T10:49:28.931 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8" 2021-12-08T10:49:28.931 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2021-12-08T10:49:28.931 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-08T10:49:28.931 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2021-12-08T10:49:28.931 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-08T10:49:28.932 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2021-12-08T10:49:28.932 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-08T10:49:28.932 INFO:teuthology.orchestra.run.smithi150.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-08T10:49:28.932 INFO:teuthology.orchestra.run.smithi150.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-08T10:49:28.933 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2021-12-08T10:49:28.951 INFO:teuthology.lock.ops:Updating [smithi180.front.sepia.ceph.com]: set os type and version on server 2021-12-08T10:49:28.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi180.front.sepia.ceph.com' 2021-12-08T10:49:29.154 INFO:teuthology.orchestra.run.smithi180.stdout:x86_64 2021-12-08T10:49:29.211 INFO:teuthology.orchestra.run.smithi180.stdout:NAME="CentOS Linux" 2021-12-08T10:49:29.211 INFO:teuthology.orchestra.run.smithi180.stdout:VERSION="8" 2021-12-08T10:49:29.211 INFO:teuthology.orchestra.run.smithi180.stdout:ID="centos" 2021-12-08T10:49:29.211 INFO:teuthology.orchestra.run.smithi180.stdout:ID_LIKE="rhel fedora" 2021-12-08T10:49:29.212 INFO:teuthology.orchestra.run.smithi180.stdout:VERSION_ID="8" 2021-12-08T10:49:29.212 INFO:teuthology.orchestra.run.smithi180.stdout:PLATFORM_ID="platform:el8" 2021-12-08T10:49:29.212 INFO:teuthology.orchestra.run.smithi180.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-08T10:49:29.213 INFO:teuthology.orchestra.run.smithi180.stdout:ANSI_COLOR="0;31" 2021-12-08T10:49:29.213 INFO:teuthology.orchestra.run.smithi180.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-08T10:49:29.213 INFO:teuthology.orchestra.run.smithi180.stdout:HOME_URL="https://centos.org/" 2021-12-08T10:49:29.213 INFO:teuthology.orchestra.run.smithi180.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-08T10:49:29.213 INFO:teuthology.orchestra.run.smithi180.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-08T10:49:29.214 INFO:teuthology.orchestra.run.smithi180.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-08T10:49:29.215 INFO:teuthology.lock.ops:Updating smithi180.front.sepia.ceph.com on lock server 2021-12-08T10:49:29.314 INFO:teuthology.dispatcher.supervisor:Running job 6549402 2021-12-08T10:49:29.315 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-12-07_03:15:03-fs-master-distro-default-smithi/6549402 --name teuthology-2021-12-07_03:15:03-fs-master-distro-default-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} -- /home/teuthworker/archive/teuthology-2021-12-07_03:15:03-fs-master-distro-default-smithi/6549402/orig.config.yaml 2021-12-08T10:49:29.318 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-12-07_03:15:03-fs-master-distro-default-smithi/6549402 2021-12-08T10:49:29.320 INFO:teuthology.dispatcher.supervisor:Job PID: 19683 2021-12-08T10:49:29.320 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-08T22:49:50.622 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-08T22:49:50.908 INFO:teuthology.kill:Killing Pids: {19683} 2021-12-08T22:49:52.332 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2021-12-08T22:49:52.333 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2021-12-08T22:49:52.333 INFO:teuthology.task.internal:roles: ubuntu@smithi180.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2021-12-08T22:49:52.334 INFO:teuthology.misc:Compressing logs... 2021-12-08T22:49:52.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2021-12-08T22:49:52.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2021-12-08T22:49:52.907 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi180.front.sepia.ceph.com' 2021-12-08T22:49:53.501 INFO:teuthology.misc:Compressing logs... 2021-12-08T22:49:53.775 INFO:teuthology.misc:Compressing logs... 2021-12-08T22:57:54.958 INFO:teuthology.kill:No teuthology processes running 2021-12-08T22:57:56.345 INFO:teuthology.kill:Nuking machines: ['smithi148', 'smithi150', 'smithi180'] 2021-12-08T22:57:57.350 INFO:teuthology.kill:2021-12-08 22:57:57,350.350 INFO:teuthology.nuke:targets: 2021-12-08T22:57:57.351 INFO:teuthology.kill: smithi148.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlPr7LGW9FVrEJGaogxPx8ITW5IoIJt6bTHnTIN913mmdYeIhVctorZW/qA2bMs3WqvcDi8m5jjZYsCwuuSv1bc9DJ9P9Gc72SJkY10zWfbWQLTUNyuFZq/hwdzAdnqBeqnZX1tBYOYHBi7zi6TKwW2iA75X2f98KbSoyK9FPJyJWs6lfOzq2o9mI8gXUOqQf96ClHYSdv1fo2umN9TuZs8Lf7yQDvFs2Lwy70nC0EPU526T1Bbd7FIomUWCWGdzO+jjPb7hHNApecm9abOjq5pMMNJbf1pyjN8Gn4ByQPps4TSO8AJ6GlgeekypNrsAUDRYoP2Nu7QSSYuTF0ftPfb+B4GlZ+ReDyuxKrw/K90CSay7hKuqPoxnXya0Cp2tEcFfcj7W62jQHKLDGJ4YxiYCELgdoibwYvTRALwGQQhhnO0qbsWSZhHa3OGv2N22Rq8oZgpn9HODoW3jQa63PPfJqR2j1It9Kfx0SxQbBzWbey5YNr25odWNrcBs32PFk= 2021-12-08T22:57:57.352 INFO:teuthology.kill: smithi150.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCX/ZeTfiO3I8648W1GxZLQrXkad/Veqnq1OxiQN596ytEg2ONWOcr1zvDNh4DBnfVrrPCbNUD6Y3xu1Jk4jMXPdevEng29iIdR+AdnC1iW2wLFt1Ts+3r2o0VxH+voUEGEzQ39eK9p+dskJwuPrGn8bwW+xnBjC5wZfUJYxYMKAuF1VYXvnLBGiZOzRo1Ku4ddPpwR8j5Y+4c/uzV9v9LlSR9OC2iu1xohtmbalFJaZlPMoP3CTBxjdpZroCV75zUyFAtPnfLdC/V9wb2cup/uKzdoThegWpe+SLphCKkzKMO96GHTLpo+i+xjrdyDNTHIZUto/7+Lh08+sp3Mgs+VA54+3XNTnx4KsFxJ5tLGSldx2+QCReiqnTvowb3jPQddJBEKtd4yuoc+SsmpX9gMVialg+cfkgy5u97tAal4HDslJSO64sDzoZKyv39TbTEr5G+NWgsslofNPXYwnc0tRpli4O2+8GSMrTvhdCExxeDxvQqW5nXv73+xqmYrrD0= 2021-12-08T22:57:57.352 INFO:teuthology.kill: smithi180.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCx6847q8Bj8cZHh2OlwsCw/MAjEW4aaxKMY/xU0EVmj2ujo+YsFB9ojfGHj5xOzx4bR7q4Nm3agIVUKInEZNfU/plX87pGtATeAeXtB8mwOvIg/Ft3G5ucpR04wAO3WaUnH0lknWrm1Rh+KRnRBCtssEXjl7hEzpCOWBb7tBV0FIVXr1Ew0LKEf3695wmpDb6zK/hNM3RR6AF0xF8WhCTx3GVWgboDz2xhF0IT+mGqoZ4WMDQ+JMG5N93USuAookAB73zGFzebKzXgmidZMtkL3MZmWArfKCXe+TugZ9Mr43oeOAMm+T/shUgiKswyVU28RBktHWCD4nx6SKfovjPjhCF0ufdgN9v1ex/JD7zphP0xWieBC/x6cCJhUWGtXsOYwH1WpzHhm2/RW9/r6q9arY1EmGBWCJkx8IUozY7QJH2OSHYZIjOjqpY15MxldAH02eprQFrcp8iR6TGFG/B5yZYCWMbSUrGYdBO+oGwYmTgmx2BzixwqgrVCOtIOkL0= 2021-12-08T22:57:57.381 INFO:teuthology.kill:2021-12-08 22:57:57,381.381 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-08T22:57:57.384 INFO:teuthology.kill:2021-12-08 22:57:57,384.384 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-08T22:57:57.393 INFO:teuthology.kill:2021-12-08 22:57:57,393.393 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-08T22:57:57.456 INFO:teuthology.kill:2021-12-08 22:57:57,456.456 INFO:teuthology.orchestra.console:Power off smithi150 2021-12-08T22:57:57.472 INFO:teuthology.kill:2021-12-08 22:57:57,471.471 INFO:teuthology.orchestra.console:Power off smithi180 2021-12-08T22:57:57.483 INFO:teuthology.kill:2021-12-08 22:57:57,483.483 INFO:teuthology.orchestra.console:Power off smithi148 2021-12-08T22:58:05.840 INFO:teuthology.kill:2021-12-08 22:58:05,839.839 INFO:teuthology.orchestra.console:Power off for smithi150 completed 2021-12-08T22:58:05.847 INFO:teuthology.kill:2021-12-08 22:58:05,846.846 INFO:teuthology.orchestra.console:Power off for smithi180 completed 2021-12-08T22:58:05.996 INFO:teuthology.kill:2021-12-08 22:58:05,995.995 INFO:teuthology.lock.ops:unlocked: smithi150.front.sepia.ceph.com 2021-12-08T22:58:06.000 INFO:teuthology.kill:2021-12-08 22:58:06,000.000 INFO:teuthology.lock.ops:unlocked: smithi180.front.sepia.ceph.com 2021-12-08T22:58:09.963 INFO:teuthology.kill:2021-12-08 22:58:09,963.963 INFO:teuthology.orchestra.console:Power off for smithi148 completed 2021-12-08T22:58:10.106 INFO:teuthology.kill:2021-12-08 22:58:10,106.106 INFO:teuthology.lock.ops:unlocked: smithi148.front.sepia.ceph.com 2021-12-08T23:00:10.433 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-08T23:00:10.578 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi148 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-08_21:00:15-rados-wip-yuri10-testing-2021-12-08-0827-octopus-distro-default-smithi/6553512 2021-12-08T23:00:10.578 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi150 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-08_21:00:15-rados-wip-yuri10-testing-2021-12-08-0827-octopus-distro-default-smithi/6553514 2021-12-08T23:00:10.579 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi180 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-08_21:00:15-rados-wip-yuri10-testing-2021-12-08-0827-octopus-distro-default-smithi/6553512