2022-09-18T14:38:15.137 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-18T14:38:15.273 INFO:teuthology.lock.ops:Start node 'smithi084.front.sepia.ceph.com' reimaging 2022-09-18T14:38:15.275 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: reset os type and version on server 2022-09-18T14:38:15.275 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2022-09-18T14:38:15.294 INFO:teuthology.lock.ops:Node 'smithi084.front.sepia.ceph.com' reimaging is complete 2022-09-18T14:38:15.294 INFO:teuthology.lock.ops:Start node 'smithi163.front.sepia.ceph.com' reimaging 2022-09-18T14:38:15.295 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: reset os type and version on server 2022-09-18T14:38:15.295 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2022-09-18T14:38:15.316 INFO:teuthology.lock.ops:Node 'smithi163.front.sepia.ceph.com' reimaging is complete 2022-09-18T14:38:15.509 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of rhel 8.6 2022-09-18T14:38:15.540 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of rhel 8.6 2022-09-18T14:38:15.817 INFO:teuthology.orchestra.console:Power off smithi084 2022-09-18T14:38:15.860 INFO:teuthology.orchestra.console:Power off smithi163 2022-09-18T14:38:28.310 INFO:teuthology.orchestra.console:Power off for smithi084 completed 2022-09-18T14:38:28.336 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2022-09-18T14:38:28.412 INFO:teuthology.orchestra.console:Power on smithi084 2022-09-18T14:38:28.442 INFO:teuthology.orchestra.console:Power on smithi163 2022-09-18T14:38:32.673 INFO:teuthology.orchestra.console:Power on for smithi084 completed 2022-09-18T14:38:32.689 INFO:teuthology.orchestra.console:Power on for smithi163 completed 2022-09-18T14:38:32.776 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish 2022-09-18T14:38:32.790 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish 2022-09-18T14:42:44.588 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-18T14:43:09.832 INFO:teuthology.provision.fog.smithi084:Node is ready 2022-09-18T14:43:09.851 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com 2022-09-18T14:43:09.908 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084 2022-09-18T14:43:10.260 INFO:teuthology.provision.fog.smithi084:Deploy complete! 2022-09-18T14:43:59.341 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-18T14:43:59.569 INFO:teuthology.provision.fog.smithi163:Node is ready 2022-09-18T14:43:59.626 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163 2022-09-18T14:43:59.684 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163 2022-09-18T14:44:00.018 INFO:teuthology.provision.fog.smithi163:Deploy complete! 2022-09-18T14:44:00.216 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com 2022-09-18T14:44:00.217 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-18T14:44:00.238 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com 2022-09-18T14:44:00.239 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-18T14:44:00.257 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server 2022-09-18T14:44:00.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2022-09-18T14:44:00.525 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2022-09-18T14:44:00.582 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Red Hat Enterprise Linux" 2022-09-18T14:44:00.582 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8.6 (Ootpa)" 2022-09-18T14:44:00.583 INFO:teuthology.orchestra.run.smithi084.stdout:ID="rhel" 2022-09-18T14:44:00.583 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="fedora" 2022-09-18T14:44:00.583 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8.6" 2022-09-18T14:44:00.583 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8" 2022-09-18T14:44:00.584 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-18T14:44:00.584 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2022-09-18T14:44:00.584 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-18T14:44:00.584 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.redhat.com/" 2022-09-18T14:44:00.585 INFO:teuthology.orchestra.run.smithi084.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-18T14:44:00.585 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-18T14:44:00.585 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-09-18T14:44:00.586 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-18T14:44:00.586 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-18T14:44:00.586 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-18T14:44:00.587 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-18T14:44:00.588 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2022-09-18T14:44:00.609 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server 2022-09-18T14:44:00.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2022-09-18T14:44:00.808 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2022-09-18T14:44:00.869 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Red Hat Enterprise Linux" 2022-09-18T14:44:00.869 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8.6 (Ootpa)" 2022-09-18T14:44:00.869 INFO:teuthology.orchestra.run.smithi163.stdout:ID="rhel" 2022-09-18T14:44:00.870 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="fedora" 2022-09-18T14:44:00.870 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8.6" 2022-09-18T14:44:00.870 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8" 2022-09-18T14:44:00.870 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-18T14:44:00.870 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2022-09-18T14:44:00.871 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-18T14:44:00.871 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.redhat.com/" 2022-09-18T14:44:00.871 INFO:teuthology.orchestra.run.smithi163.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-18T14:44:00.871 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-18T14:44:00.872 INFO:teuthology.orchestra.run.smithi163.stdout: 2022-09-18T14:44:00.872 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-18T14:44:00.872 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-18T14:44:00.872 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-18T14:44:00.872 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-18T14:44:00.873 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2022-09-18T14:44:00.973 INFO:teuthology.dispatcher.supervisor:Running job 7033915 2022-09-18T14:44:00.973 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-09-15_14:29:45-rados:thrash-erasure-code-main-distro-default-smithi/7033915 --name lflores-2022-09-15_14:29:45-rados:thrash-erasure-code-main-distro-default-smithi --description rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} -- /home/teuthworker/archive/lflores-2022-09-15_14:29:45-rados:thrash-erasure-code-main-distro-default-smithi/7033915/orig.config.yaml 2022-09-18T14:44:00.978 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-09-15_14:29:45-rados:thrash-erasure-code-main-distro-default-smithi/7033915 2022-09-18T14:44:00.979 INFO:teuthology.dispatcher.supervisor:Job PID: 598 2022-09-18T14:44:00.979 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-19T02:44:24.437 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-19T02:44:24.591 INFO:teuthology.kill:Killing Pids: {598} 2022-09-19T02:44:25.987 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-19T02:44:25.988 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-19T02:44:25.989 INFO:teuthology.misc:Compressing logs... 2022-09-19T02:44:25.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2022-09-19T02:44:26.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2022-09-19T02:44:26.542 INFO:teuthology.misc:Compressing logs... 2022-09-19T02:44:27.606 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-19T02:44:27.631 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-19T02:44:28.265 INFO:teuthology.misc:Compressing logs... 2022-09-19T02:46:00.492 INFO:teuthology.kill:No teuthology processes running 2022-09-19T02:46:01.779 INFO:teuthology.kill:Nuking machines: ['smithi084', 'smithi163'] 2022-09-19T02:46:02.878 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-19T02:46:02.879 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-19T02:46:02.879 INFO:teuthology.kill:2022-09-19 02:46:02,878.878 INFO:teuthology.nuke:targets: 2022-09-19T02:46:02.880 INFO:teuthology.kill: smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoYQnrZ6+MiXEcW76RHDKU63cweMF4cIeLUqu00PUzzN+QwLmQKWDdhVsVyU75stNONTNb3qMe27ENPmanRpgtwECjfI2VyrIzLo/2iS0bG45IchUQq9Z8Tfp09VmMdAkdHuJkLiA3y93qeF3hAR/KMYrDYt2A7FIixqxsbg2+bNsxiQTpqtmlFtBFRW7RO4EcZSstfE5oBmwFD/5m1kPx7x/w910SXD/NHTh+RHd2WuP/l21sLWpKqryIBr0K2mW9MwHgp0cGB12VJumtJCz0ZhpuaEKfaKJdgQajhbcWfpUWkxlxLbpbYpZ2cTd5Uau9TXFc4wGLgGXIJD0VvERUnHiBvMbuHRochQ7gopDdpGV95FUeX1z4F5xADI4K36vHyzWRhSH8uWfkdjE3iKiwwE8QBFaQdGxbD3diZ+q81gR6KKY/S5X20rBEHfxxeUWwzMWCNWRyk3ow08zBRlyqWFvyRdYoa/R8bXfuYwzppptbl5L36kYrvVHBQ7MZddU= 2022-09-19T02:46:02.880 INFO:teuthology.kill: smithi163.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgk2YDmE2Xfg++2+VKv3/HETIAbd4F+JYMFwWw+Djc3Ul9x2z1QdrjcuWfRdYq7JwKAcz6vTapKyT+OwmJGAuUGcAh57pH++0v1gMlM3PUM4xF1SfMbNFfMf/V7EUy8jUySTDo3m6kvn8KWPzyNUnyYbpCHo06H1s2mA8MzN16INLc+jT+ZUCzf724nqLNhD7Oo/BpUquEsCAvjVL3h2U5sFUImohD/qgTnH6nw9oqYhTuvncO4q0HaMdMohWHXPT48qpN4GfWptPm1qgT79IzjaQd4OQrkF71TmzfRKpmjaWNiWLx7tlbuBbOcT62SMt78znWx17u3s9OzaKDsul3JAQTvflWO/zX2YDvv7RbG5XWaLF2+VdV9zEPqbtQFoeNghBpnXtk7EVBIreW9kcvKsJi2I+ey1C5P/DDV0CKJ9nSvcz2vZRmioXXNZZpnBtmLaT411G/6Dsj2SUfOlWgY4RB74FtEw2eonq8BgcMmrqtnbEN6uqdZMm/PI41sus= 2022-09-19T02:46:02.880 INFO:teuthology.kill:2022-09-19 02:46:02,878.878 INFO:teuthology.nuke:Checking targets against current locks 2022-09-19T02:46:02.933 INFO:teuthology.kill:2022-09-19 02:46:02,933.933 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-19T02:46:02.956 INFO:teuthology.kill:2022-09-19 02:46:02,956.956 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-19T02:46:03.016 INFO:teuthology.kill:2022-09-19 02:46:03,016.016 INFO:teuthology.orchestra.console:Power off smithi084 2022-09-19T02:46:03.054 INFO:teuthology.kill:2022-09-19 02:46:03,054.054 INFO:teuthology.orchestra.console:Power off smithi163 2022-09-19T02:46:15.501 INFO:teuthology.kill:2022-09-19 02:46:15,500.500 INFO:teuthology.orchestra.console:Power off for smithi084 completed 2022-09-19T02:46:15.523 INFO:teuthology.kill:2022-09-19 02:46:15,523.523 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2022-09-19T02:46:15.652 INFO:teuthology.kill:2022-09-19 02:46:15,651.651 INFO:teuthology.lock.ops:unlocked: smithi084.front.sepia.ceph.com 2022-09-19T02:46:15.675 INFO:teuthology.kill:2022-09-19 02:46:15,674.674 INFO:teuthology.lock.ops:unlocked: smithi163.front.sepia.ceph.com 2022-09-19T02:48:15.994 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-19T02:48:16.122 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi084 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-09-19_02:01:03-rbd-quincy-distro-default-smithi/7037467 2022-09-19T02:48:16.122 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi163 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-09-19_02:01:03-rbd-quincy-distro-default-smithi/7037463