2021-12-25T06:07:55.622 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-25T06:07:55.779 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging 2021-12-25T06:07:55.780 INFO:teuthology.lock.ops:Updating [ubuntu@smithi174.front.sepia.ceph.com]: reset os type and version on server 2021-12-25T06:07:55.780 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2021-12-25T06:07:55.799 INFO:teuthology.lock.ops:Node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging is complete 2021-12-25T06:07:55.799 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi190.front.sepia.ceph.com' reimaging 2021-12-25T06:07:55.799 INFO:teuthology.lock.ops:Updating [ubuntu@smithi190.front.sepia.ceph.com]: reset os type and version on server 2021-12-25T06:07:55.800 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2021-12-25T06:07:55.819 INFO:teuthology.lock.ops:Node 'ubuntu@smithi190.front.sepia.ceph.com' reimaging is complete 2021-12-25T06:07:55.820 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging 2021-12-25T06:07:55.820 INFO:teuthology.lock.ops:Updating [ubuntu@smithi196.front.sepia.ceph.com]: reset os type and version on server 2021-12-25T06:07:55.820 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2021-12-25T06:07:55.839 INFO:teuthology.lock.ops:Node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging is complete 2021-12-25T06:07:55.994 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 8.3 2021-12-25T06:07:56.031 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of centos 8.3 2021-12-25T06:07:56.056 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of centos 8.3 2021-12-25T06:07:56.343 INFO:teuthology.orchestra.console:Power off smithi174 2021-12-25T06:07:56.382 INFO:teuthology.orchestra.console:Power off smithi190 2021-12-25T06:07:56.418 INFO:teuthology.orchestra.console:Power off smithi196 2021-12-25T06:08:04.720 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2021-12-25T06:08:04.821 INFO:teuthology.orchestra.console:Power on smithi174 2021-12-25T06:08:08.862 INFO:teuthology.orchestra.console:Power off for smithi190 completed 2021-12-25T06:08:08.886 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2021-12-25T06:08:08.963 INFO:teuthology.orchestra.console:Power on smithi190 2021-12-25T06:08:08.994 INFO:teuthology.orchestra.console:Power on smithi196 2021-12-25T06:08:13.305 INFO:teuthology.orchestra.console:Power on for smithi174 completed 2021-12-25T06:08:13.406 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2021-12-25T06:08:17.333 INFO:teuthology.orchestra.console:Power on for smithi190 completed 2021-12-25T06:08:17.356 INFO:teuthology.orchestra.console:Power on for smithi196 completed 2021-12-25T06:08:17.435 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish 2021-12-25T06:08:17.457 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2021-12-25T06:12:15.434 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-25T06:12:35.692 INFO:teuthology.provision.fog.smithi190:Node is ready 2021-12-25T06:12:35.712 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190.front.sepia.ceph.com 2021-12-25T06:12:35.769 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.15.190 smithi190.front.sepia.ceph.com smithi190 2021-12-25T06:12:36.122 INFO:teuthology.provision.fog.smithi190:Deploy complete! 2021-12-25T06:13:18.428 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-25T06:13:18.719 INFO:teuthology.provision.fog.smithi174:Node is ready 2021-12-25T06:13:18.774 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2021-12-25T06:13:18.829 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2021-12-25T06:13:19.156 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2021-12-25T06:13:22.470 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-25T06:13:22.649 INFO:teuthology.provision.fog.smithi196:Node is ready 2021-12-25T06:13:22.704 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com 2021-12-25T06:13:22.758 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196 2021-12-25T06:13:23.138 INFO:teuthology.provision.fog.smithi196:Deploy complete! 2021-12-25T06:13:23.388 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2021-12-25T06:13:23.389 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-25T06:13:23.407 INFO:teuthology.lock.ops:Checking smithi190.front.sepia.ceph.com 2021-12-25T06:13:23.407 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-25T06:13:23.423 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com 2021-12-25T06:13:23.423 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-25T06:13:23.440 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2021-12-25T06:13:23.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2021-12-25T06:13:23.652 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2021-12-25T06:13:23.708 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Linux" 2021-12-25T06:13:23.708 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="8" 2021-12-25T06:13:23.709 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2021-12-25T06:13:23.709 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2021-12-25T06:13:23.709 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="8" 2021-12-25T06:13:23.709 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el8" 2021-12-25T06:13:23.710 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-25T06:13:23.710 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2021-12-25T06:13:23.710 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-25T06:13:23.710 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2021-12-25T06:13:23.711 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-25T06:13:23.711 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-25T06:13:23.711 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-25T06:13:23.712 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2021-12-25T06:13:23.731 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: set os type and version on server 2021-12-25T06:13:23.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2021-12-25T06:13:23.990 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2021-12-25T06:13:24.046 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Linux" 2021-12-25T06:13:24.047 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8" 2021-12-25T06:13:24.047 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2021-12-25T06:13:24.047 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2021-12-25T06:13:24.048 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8" 2021-12-25T06:13:24.048 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2021-12-25T06:13:24.048 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-25T06:13:24.049 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2021-12-25T06:13:24.049 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-25T06:13:24.049 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2021-12-25T06:13:24.050 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-25T06:13:24.050 INFO:teuthology.orchestra.run.smithi190.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-25T06:13:24.050 INFO:teuthology.orchestra.run.smithi190.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-25T06:13:24.051 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2021-12-25T06:13:24.069 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server 2021-12-25T06:13:24.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2021-12-25T06:13:24.280 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2021-12-25T06:13:24.334 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Linux" 2021-12-25T06:13:24.334 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8" 2021-12-25T06:13:24.334 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2021-12-25T06:13:24.335 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2021-12-25T06:13:24.335 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8" 2021-12-25T06:13:24.335 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2021-12-25T06:13:24.335 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-25T06:13:24.336 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2021-12-25T06:13:24.336 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-25T06:13:24.336 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2021-12-25T06:13:24.336 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-25T06:13:24.336 INFO:teuthology.orchestra.run.smithi196.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-25T06:13:24.337 INFO:teuthology.orchestra.run.smithi196.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-25T06:13:24.337 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2021-12-25T06:13:24.449 INFO:teuthology.dispatcher.supervisor:Running job 6574218 2021-12-25T06:13:24.449 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-12-21_03:15:03-fs-master-distro-default-smithi/6574218 --name teuthology-2021-12-21_03:15:03-fs-master-distro-default-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} -- /home/teuthworker/archive/teuthology-2021-12-21_03:15:03-fs-master-distro-default-smithi/6574218/orig.config.yaml 2021-12-25T06:13:24.452 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-12-21_03:15:03-fs-master-distro-default-smithi/6574218 2021-12-25T06:13:24.453 INFO:teuthology.dispatcher.supervisor:Job PID: 29401 2021-12-25T06:13:24.454 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-25T18:14:18.060 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-25T18:14:18.273 INFO:teuthology.kill:Killing Pids: {29401} 2021-12-25T18:14:19.507 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2021-12-25T18:14:19.508 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2021-12-25T18:14:19.508 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2021-12-25T18:14:19.509 INFO:teuthology.misc:Compressing logs... 2021-12-25T18:14:19.509 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2021-12-25T18:14:19.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2021-12-25T18:14:20.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2021-12-25T18:14:20.811 INFO:teuthology.misc:Compressing logs... 2021-12-25T18:14:21.071 INFO:teuthology.misc:Compressing logs... 2021-12-25T18:26:21.197 INFO:teuthology.kill:No teuthology processes running 2021-12-25T18:26:22.402 INFO:teuthology.kill:Nuking machines: ['smithi174', 'smithi190', 'smithi196'] 2021-12-25T18:26:23.363 INFO:teuthology.kill:2021-12-25 18:26:23,362.362 INFO:teuthology.nuke:targets: 2021-12-25T18:26:23.363 INFO:teuthology.kill: smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8AOfj4mwIKrSAPrcCwF0ZxPM/SLSv1OqabZuyZsHlMBP3CHBeVu7jNRMctCVeOl4eRvNVxdtYhb1bHpuftxcDj/KuhzU51rGaXGDmvKeE7EFLbKBO81DRoAJJZu97R8WBKap2a8/tq7CE6uJ/zZelJoDKOsMSV6XdndvmoWOnJPVdQe4EtPMjyRVItFcksiGVgjqjIFH0hnLSyIaAy/pc9U6MeZRjliGu1sPqYtZkbUmZHmZyjV6xSNXoThHE+yewt5w4rjYo0dH3G2qb4iSsVUq04UaGQQsgyXDsY1v03PNCTZgq42s3DheCOlQNzjxp9owmxbeqsVmOXCILwuhs20sb3Z8Z3Hhu/LkZe4B4hajDkrp8y4pmussK3yg9wnHWFB8ZwwEUW/doyi98zU574R9ujDG44ImQF1LNSMHW6TrKyPyvOA6gWbR1LkwZ6zCoSE8fXBMMrMzMB0E+j2Z5Su/EHCFMtuiyztDJMEs00MV8uzsX/tno//1sIHmhqUM= 2021-12-25T18:26:23.364 INFO:teuthology.kill: smithi190.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvRkufPHQoNLYSKe4z8EtOHGdEwMaTct2K74xjmZJ6gfaqbZY4/oie65YaG5U822BKPhmliNow5GKId99nRbSP4tG8/qc2n7mF/oT5qp0eQNgd+jMqmElkkj1hQuC+2UlylLEqnSBR7OUf5i0YcD4q5NnzeNsXclw+YG4WEFVn7xTK3jWNlIhF/62wbl68yc7U3vkhWZFpYPtKa1cNJUNQaPsbnu/oJke+hIygxvN6MUK9nGWWDuNTgx1mt5bupgsizcI1qLy2LtkSxXi4LY0p0MrgrXLTsehg+zr8lt5V71qfMMp5aHNFdFNjulkebF7q7TqveQLaK73ExW7Bmgsl3DP2dJTAfZ83QZVbZRkp4b6+F+uK/cNF72qZv5q1FNCe6Aq9wLKEOYo1rcxz1BhJPhfzoGCAZmzl/pom0kxKWbGKmZWsZCPPDfQ3vNHpOLy2L7mzuXVzsixrvA27LRAOUYQ110ktXCfQV5QaB+vh+03UUXJQJ0xbM5HqJDsvTmU= 2021-12-25T18:26:23.364 INFO:teuthology.kill: smithi196.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8NU1Oa4A+QKakmDRCEJbzWeByKA1yN45lGwZdltjvC9Y42LAonb2wZH3QCnVNR+zHyPStb+CcNv73r9Iu1qP6vXLMA7riJpy5XT3/+gwCwWPpp/pUbDgET7NaPUNBXa8uFXt0/81TiQ3ObY1xfE32cDELqbmdpjMCWiWHbsmeCv3Ct9fdPQSmhFDLzNd3oR08TKSVxkFdY/GG4rN1HoaVvz93QiTxk4oG+w208cvvkCd0y/xPENj9/xYkNtc6b3UmB9DZhKAvYM45twCGVgGt32xYs/SpLwIWB9kHtXCyZl005fFiTSjlayO/pVK9nud8wgAcZjgLbqYB/v08BwnnDj4eI9/qpVaoaqEwJKMWfM0r75ZCnA1AsiLVIMkzRFtMu4sBs8YM4XpX3iLMB0+RHtF7AgjdxN+ZtiswWeiDpicBJfOCkFOeBwVFgf5W1r8nDtiTEzq2faws9zmgg71sG1mile2LonpA6ijQwXm3agWpY8xD3UYBW6Chw75sJC8= 2021-12-25T18:26:23.393 INFO:teuthology.kill:2021-12-25 18:26:23,392.392 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-25T18:26:23.397 INFO:teuthology.kill:2021-12-25 18:26:23,397.397 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-25T18:26:23.399 INFO:teuthology.kill:2021-12-25 18:26:23,398.398 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-25T18:26:23.480 INFO:teuthology.kill:2021-12-25 18:26:23,480.480 INFO:teuthology.orchestra.console:Power off smithi196 2021-12-25T18:26:23.494 INFO:teuthology.kill:2021-12-25 18:26:23,493.493 INFO:teuthology.orchestra.console:Power off smithi174 2021-12-25T18:26:23.504 INFO:teuthology.kill:2021-12-25 18:26:23,504.504 INFO:teuthology.orchestra.console:Power off smithi190 2021-12-25T18:26:31.863 INFO:teuthology.kill:2021-12-25 18:26:31,863.863 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2021-12-25T18:26:32.007 INFO:teuthology.kill:2021-12-25 18:26:32,006.006 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com 2021-12-25T18:26:35.969 INFO:teuthology.kill:2021-12-25 18:26:35,968.968 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2021-12-25T18:26:35.977 INFO:teuthology.kill:2021-12-25 18:26:35,977.977 INFO:teuthology.orchestra.console:Power off for smithi190 completed 2021-12-25T18:26:36.121 INFO:teuthology.kill:2021-12-25 18:26:36,121.121 INFO:teuthology.lock.ops:unlocked: smithi196.front.sepia.ceph.com 2021-12-25T18:26:36.145 INFO:teuthology.kill:2021-12-25 18:26:36,145.145 INFO:teuthology.lock.ops:unlocked: smithi190.front.sepia.ceph.com 2021-12-25T18:28:36.443 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-25T18:28:36.583 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi174 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-25_16:12:31-rados-wip-yuri-master-12.24.21-distro-default-smithi/6584128 2021-12-25T18:28:36.583 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi190 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-25_16:12:31-rados-wip-yuri-master-12.24.21-distro-default-smithi/6584127 2021-12-25T18:28:36.584 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi196 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-25_16:12:31-rados-wip-yuri-master-12.24.21-distro-default-smithi/6584127