2021-11-23T19:07:06.692 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-23T19:07:06.839 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi122.front.sepia.ceph.com' reimaging 2021-11-23T19:07:06.840 INFO:teuthology.lock.ops:Updating [ubuntu@smithi122.front.sepia.ceph.com]: reset os type and version on server 2021-11-23T19:07:06.841 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2021-11-23T19:07:06.863 INFO:teuthology.lock.ops:Node 'ubuntu@smithi122.front.sepia.ceph.com' reimaging is complete 2021-11-23T19:07:06.863 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging 2021-11-23T19:07:06.864 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server 2021-11-23T19:07:06.865 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2021-11-23T19:07:06.887 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete 2021-11-23T19:07:07.076 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of centos 8.3 2021-11-23T19:07:07.106 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of centos 8.3 2021-11-23T19:07:07.455 INFO:teuthology.orchestra.console:Power off smithi122 2021-11-23T19:07:07.469 INFO:teuthology.orchestra.console:Power off smithi185 2021-11-23T19:07:15.835 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2021-11-23T19:07:15.936 INFO:teuthology.orchestra.console:Power on smithi122 2021-11-23T19:07:19.957 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2021-11-23T19:07:20.059 INFO:teuthology.orchestra.console:Power on smithi185 2021-11-23T19:07:24.316 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2021-11-23T19:07:24.318 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2021-11-23T19:07:24.419 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2021-11-23T19:07:24.421 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2021-11-23T19:09:55.902 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: Authentication failed. 2021-11-23T19:12:09.190 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-23T19:12:09.493 INFO:teuthology.provision.fog.smithi122:Node is ready 2021-11-23T19:12:09.508 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122.front.sepia.ceph.com 2021-11-23T19:12:09.563 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122 smithi122.front.sepia.ceph.com smithi122 2021-11-23T19:12:09.879 INFO:teuthology.provision.fog.smithi122:Deploy complete! 2021-11-23T19:12:29.242 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-23T19:12:29.554 INFO:teuthology.provision.fog.smithi185:Node is ready 2021-11-23T19:12:29.633 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2021-11-23T19:12:29.703 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2021-11-23T19:12:30.096 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2021-11-23T19:12:30.295 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com 2021-11-23T19:12:30.296 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-23T19:12:30.314 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2021-11-23T19:12:30.315 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-23T19:12:30.333 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server 2021-11-23T19:12:30.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2021-11-23T19:12:30.605 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2021-11-23T19:12:30.660 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Linux" 2021-11-23T19:12:30.660 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="8" 2021-11-23T19:12:30.660 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2021-11-23T19:12:30.661 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2021-11-23T19:12:30.661 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="8" 2021-11-23T19:12:30.661 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el8" 2021-11-23T19:12:30.661 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Linux 8" 2021-11-23T19:12:30.662 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2021-11-23T19:12:30.662 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-23T19:12:30.662 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2021-11-23T19:12:30.662 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-11-23T19:12:30.663 INFO:teuthology.orchestra.run.smithi122.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-11-23T19:12:30.663 INFO:teuthology.orchestra.run.smithi122.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-11-23T19:12:30.664 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2021-11-23T19:12:30.682 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2021-11-23T19:12:30.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2021-11-23T19:12:30.943 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2021-11-23T19:12:30.998 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="CentOS Linux" 2021-11-23T19:12:30.998 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8" 2021-11-23T19:12:30.998 INFO:teuthology.orchestra.run.smithi185.stdout:ID="centos" 2021-11-23T19:12:30.999 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="rhel fedora" 2021-11-23T19:12:30.999 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8" 2021-11-23T19:12:30.999 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8" 2021-11-23T19:12:31.000 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="CentOS Linux 8" 2021-11-23T19:12:31.000 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31" 2021-11-23T19:12:31.000 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-23T19:12:31.000 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://centos.org/" 2021-11-23T19:12:31.001 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-11-23T19:12:31.001 INFO:teuthology.orchestra.run.smithi185.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-11-23T19:12:31.001 INFO:teuthology.orchestra.run.smithi185.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-11-23T19:12:31.002 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2021-11-23T19:12:31.105 INFO:teuthology.dispatcher.supervisor:Running job 6522438 2021-11-23T19:12:31.106 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522438 --name yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522438/orig.config.yaml 2021-11-23T19:12:31.109 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522438 2021-11-23T19:12:31.110 INFO:teuthology.dispatcher.supervisor:Job PID: 10455 2021-11-23T19:12:31.111 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-24T07:12:52.517 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-24T07:12:52.722 INFO:teuthology.kill:Killing Pids: {10455} 2021-11-24T07:12:53.924 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-11-24T07:12:53.925 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-11-24T07:12:53.926 INFO:teuthology.misc:Compressing logs... 2021-11-24T07:12:53.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2021-11-24T07:12:54.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2021-11-24T07:12:54.568 INFO:teuthology.misc:Compressing logs... 2021-11-24T07:12:55.534 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-24T07:12:55.828 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-24T07:12:58.781 INFO:teuthology.misc:Compressing logs... 2021-11-24T07:13:12.414 INFO:teuthology.kill:No teuthology processes running 2021-11-24T07:13:13.627 INFO:teuthology.kill:Nuking machines: ['smithi122', 'smithi185'] 2021-11-24T07:13:14.651 INFO:teuthology.kill:2021-11-24 07:13:14,651.651 INFO:teuthology.nuke:targets: 2021-11-24T07:13:14.652 INFO:teuthology.kill: smithi122.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDgieuMgQZ2v0L51X466LS970M1JjZcjsmw06yD1w8i0ICdXvSrSEztmivMCmU7QPjSLToWtvFgQKICabQvo6p3XW4co120vYXi/NTABTvxeOl/Xzg8yaipbt8wd4vU6K8Zt7iq+zemSya6hBJ3KnHA0LRBBGlZvhaPPZhj+8BPzYfSN+6ZPW0PKBh8KczJr/MJ/nsEVKeOSwcl8RVyN2TqAPzeMgWg+YYF3bQ7LsVw2vBHkL2wu765o/DlhDRExxh0ICojiIaq5rGTRFVYEc7YwIIyUkBc6tflNYSIvOI8GaEpCtzyFiepprR82BBymKFm24v5qmZpui3rfSv3DsHWp6oqcj4M5nKCKVw+FIFAatFQd3wt94gtlWYJdiJhQjk1CSRdRui1tf5QWadeCmmb71G+t2g8G+XVtdxaVnEeuz4Ka8Jqi9cYXKOWB5cIQld/7HcsLQMbgzGknS5VSIFnoCXxv8k4pTTD4zKCe+hw3cDEcJXNIMe/8pp3rSjIa8= 2021-11-24T07:13:14.653 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAFu7+YBT94/FjAe621TPsi6fEQqYuS9A/lSKcN5LzpfLwxvTvSS5bme+Q0xxphv1CHstwXVhg65khEQqyip4aUIbR7kCRYHbydSAzgCz24vUGDq+goNkst74caUZRAoFjIQl0pWYdRg8L7MPkCpRgu5/D7OTpH8CFM0sMxJprznuewPFqY366hX+X3DEq8euARGl9Wnu4Im03LRyGUyHTXSnHj9PCqIshlzNUqsZpKqp8sqaw8mGRcJvD884zLrxEBF9Av8tYV5DhAfOsJhIvdjKqoMC185sKqnE1XQSxZlnia9UATTtjIUkbpHLIWwDv8LYbce40MaL8cam5+kwO5JuR9IW90m1Vme/Yg4tUtt68FN7W4xs9RXWGBHrjgTDU8FnbYzJd7nL7Yorg9dZQ9s+wRnNOJCj46b2ix8junBE2BIBKhkswDCasa2AFDy9V3Muxq91k5AVc6CWmtOsYwV/mvwq4p54KSoBLgBXd8gQYLI6u3kCrIYxVLilVdFE= 2021-11-24T07:13:14.677 INFO:teuthology.kill:2021-11-24 07:13:14,677.677 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-24T07:13:14.681 INFO:teuthology.kill:2021-11-24 07:13:14,681.681 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-24T07:13:14.757 INFO:teuthology.kill:2021-11-24 07:13:14,757.757 INFO:teuthology.orchestra.console:Power off smithi185 2021-11-24T07:13:14.774 INFO:teuthology.kill:2021-11-24 07:13:14,774.774 INFO:teuthology.orchestra.console:Power off smithi122 2021-11-24T07:13:27.249 INFO:teuthology.kill:2021-11-24 07:13:27,249.249 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2021-11-24T07:13:27.257 INFO:teuthology.kill:2021-11-24 07:13:27,257.257 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2021-11-24T07:13:27.410 INFO:teuthology.kill:2021-11-24 07:13:27,409.409 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com 2021-11-24T07:13:27.413 INFO:teuthology.kill:2021-11-24 07:13:27,413.413 INFO:teuthology.lock.ops:unlocked: smithi122.front.sepia.ceph.com 2021-11-24T07:15:27.676 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-24T07:15:27.792 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi122 but it was locked by another job: /home/teuthworker/archive/ifed01-2021-11-23_22:21:23-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6523126 2021-11-24T07:15:27.793 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/ifed01-2021-11-23_22:21:23-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6523126