2022-09-15T04:34:38.202 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-15T04:34:38.366 INFO:teuthology.lock.ops:Start node 'smithi096.front.sepia.ceph.com' reimaging 2022-09-15T04:34:38.368 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: reset os type and version on server 2022-09-15T04:34:38.369 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2022-09-15T04:34:38.395 INFO:teuthology.lock.ops:Node 'smithi096.front.sepia.ceph.com' reimaging is complete 2022-09-15T04:34:38.395 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging 2022-09-15T04:34:38.396 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server 2022-09-15T04:34:38.396 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2022-09-15T04:34:38.419 INFO:teuthology.lock.ops:Node 'smithi169.front.sepia.ceph.com' reimaging is complete 2022-09-15T04:34:38.610 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of rhel 8.6 2022-09-15T04:34:38.636 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of rhel 8.6 2022-09-15T04:34:39.007 INFO:teuthology.orchestra.console:Power off smithi096 2022-09-15T04:34:39.025 INFO:teuthology.orchestra.console:Power off smithi169 2022-09-15T04:34:39.159 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2022-09-15T04:34:39.171 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2022-09-15T04:34:39.260 INFO:teuthology.orchestra.console:Power on smithi096 2022-09-15T04:34:39.273 INFO:teuthology.orchestra.console:Power on smithi169 2022-09-15T04:34:43.520 INFO:teuthology.orchestra.console:Power on for smithi096 completed 2022-09-15T04:34:43.621 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish 2022-09-15T04:34:47.635 INFO:teuthology.orchestra.console:Power on for smithi169 completed 2022-09-15T04:34:47.737 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish 2022-09-15T04:39:27.713 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-15T04:39:27.979 INFO:teuthology.provision.fog.smithi096:Node is ready 2022-09-15T04:39:28.041 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com 2022-09-15T04:39:28.102 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096 2022-09-15T04:39:28.449 INFO:teuthology.provision.fog.smithi096:Deploy complete! 2022-09-15T04:39:49.903 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-15T04:39:50.169 INFO:teuthology.provision.fog.smithi169:Node is ready 2022-09-15T04:39:50.227 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169 2022-09-15T04:39:50.285 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2022-09-15T04:39:50.627 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2022-09-15T04:39:50.909 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com 2022-09-15T04:39:50.910 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-15T04:39:50.933 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2022-09-15T04:39:50.934 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-15T04:39:50.962 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server 2022-09-15T04:39:50.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2022-09-15T04:39:51.227 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2022-09-15T04:39:51.286 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Red Hat Enterprise Linux" 2022-09-15T04:39:51.287 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8.6 (Ootpa)" 2022-09-15T04:39:51.287 INFO:teuthology.orchestra.run.smithi096.stdout:ID="rhel" 2022-09-15T04:39:51.287 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="fedora" 2022-09-15T04:39:51.287 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8.6" 2022-09-15T04:39:51.288 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8" 2022-09-15T04:39:51.288 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-15T04:39:51.288 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2022-09-15T04:39:51.288 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-15T04:39:51.289 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.redhat.com/" 2022-09-15T04:39:51.289 INFO:teuthology.orchestra.run.smithi096.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-15T04:39:51.289 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-15T04:39:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: 2022-09-15T04:39:51.289 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-15T04:39:51.290 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-15T04:39:51.290 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-15T04:39:51.290 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-15T04:39:51.292 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2022-09-15T04:39:51.310 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2022-09-15T04:39:51.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2022-09-15T04:39:51.530 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2022-09-15T04:39:51.589 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Red Hat Enterprise Linux" 2022-09-15T04:39:51.590 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8.6 (Ootpa)" 2022-09-15T04:39:51.590 INFO:teuthology.orchestra.run.smithi169.stdout:ID="rhel" 2022-09-15T04:39:51.590 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="fedora" 2022-09-15T04:39:51.591 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8.6" 2022-09-15T04:39:51.591 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8" 2022-09-15T04:39:51.591 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-15T04:39:51.591 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2022-09-15T04:39:51.592 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-15T04:39:51.592 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.redhat.com/" 2022-09-15T04:39:51.592 INFO:teuthology.orchestra.run.smithi169.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-15T04:39:51.592 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-15T04:39:51.592 INFO:teuthology.orchestra.run.smithi169.stdout: 2022-09-15T04:39:51.593 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-15T04:39:51.593 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-15T04:39:51.593 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-15T04:39:51.593 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-15T04:39:51.595 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2022-09-15T04:39:51.719 INFO:teuthology.dispatcher.supervisor:Running job 7033444 2022-09-15T04:39:51.720 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_pdhange@teuthology --archive /home/teuthworker/archive/pdhange-2022-09-15_04:10:46-rados-wip-pdhange-testing-distro-default-smithi/7033444 --name pdhange-2022-09-15_04:10:46-rados-wip-pdhange-testing-distro-default-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} -- /home/teuthworker/archive/pdhange-2022-09-15_04:10:46-rados-wip-pdhange-testing-distro-default-smithi/7033444/orig.config.yaml 2022-09-15T04:39:51.725 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdhange-2022-09-15_04:10:46-rados-wip-pdhange-testing-distro-default-smithi/7033444 2022-09-15T04:39:51.726 INFO:teuthology.dispatcher.supervisor:Job PID: 6245 2022-09-15T04:39:51.726 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-15T16:40:49.221 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-15T16:40:49.388 INFO:teuthology.kill:Killing Pids: {6245} 2022-09-15T16:40:50.919 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-15T16:40:50.920 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-15T16:40:50.920 INFO:teuthology.misc:Compressing logs... 2022-09-15T16:40:50.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2022-09-15T16:40:51.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2022-09-15T16:40:51.464 INFO:teuthology.misc:Compressing logs... 2022-09-15T16:40:52.544 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-15T16:40:52.548 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-15T16:40:58.259 INFO:teuthology.misc:Compressing logs... 2022-09-15T16:43:20.602 INFO:teuthology.kill:No teuthology processes running 2022-09-15T16:43:22.720 INFO:teuthology.kill:Nuking machines: ['smithi096', 'smithi169'] 2022-09-15T16:43:24.417 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-15T16:43:24.418 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-15T16:43:24.419 INFO:teuthology.kill:2022-09-15 16:43:24,417.417 INFO:teuthology.nuke:targets: 2022-09-15T16:43:24.420 INFO:teuthology.kill: smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCv9J9X+0mjCVdQJrMXDdmYTcwuq7lpeK4AFcrJp4kc6GzPjvxsReqvmCMTIrRpxeI4T8JBvrUpzWeBCaM/FVLs6YOVNwYUdlIQb6uxoi3Y081YGjwIUhtOFD+I0bQXwc/nJWkwJOo7DAv6uRb5ame4e23z4RmZ+5j22IH0AItb2OfEX5vQ1AjvOhiinafE68m7DN4tV/V/MZPdkKRo1rz7uvZG62xgWr5aiJs6kggmm0h7MPQffAbnb2fEW3O29t11Ln61GHzAajnQSf9N+PvfqE3jZ/6K1uuNfA8x2VefGo/ehgmqTptnwORKwGiC2CE8OSWOG1HtR68YX3wmAGGseQ2CAtR+qbflTCQOlz9nolj9hqmEPmVGoThHHYBfLBNrF3lsrNYtW1JZlX4yKizB5/6c09gJMwhPB82a87LkUShH0GiyTe97S+47STKOfhlkhF+qI+QzNWzyL44HJlz1NAgSnFSk49VD/sZ5x0O89Q57qOJID6a0/4DmXNOkCU= 2022-09-15T16:43:24.420 INFO:teuthology.kill: smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk6FCTxmIdr53GWXtsrvn/LcLci8LTlrvmCGhVNeOn73fMT5u3J8FZMQ9+h3eU+oHdufcvdPl7lOd7dQq3x3tMg3Z+wXrsDRGgXi9ovOoWv+XuRnWJwUCOujgqwfpzUzIZMNP2h49XdIhHo+t7FKAiGKbzfq+IaLHRHj3HV8AO5utV6UBKmq1aZ9qVcUC/rPrG6hQi6bzTEwExvt+s2iVa6SYrOpbS9v9l1ubiUx2CJ0jvI6CQeSz0loD4gIS+jeA/LdxVKoBATNxeDViTqPC780ZaC+UmLU6296QzWQCdvGH5JRJ8VT2gW0OteUoL/VtooGwy3WK34s9bQKoWxEp2No5gaJJ75mPqCanvj2nurhWnmTYyrkS6dyy9c1I/3hFujhHgnpkBUXuVqNvKd5QUyz9yu6MkvjeJKS0ih78I/Mk4INha8WYLNGC4Zakcl+Vr++U05oHwHlKq4E59f/gwf/zTzLtPozQ7gxM7tJGlxu+2NKfBx1OoSLrBV6woOY0= 2022-09-15T16:43:24.420 INFO:teuthology.kill:2022-09-15 16:43:24,417.417 INFO:teuthology.nuke:Checking targets against current locks 2022-09-15T16:43:24.524 INFO:teuthology.kill:2022-09-15 16:43:24,524.524 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-15T16:43:24.561 INFO:teuthology.kill:2022-09-15 16:43:24,561.561 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-15T16:43:24.648 INFO:teuthology.kill:2022-09-15 16:43:24,644.644 INFO:teuthology.orchestra.console:Power off smithi096 2022-09-15T16:43:24.721 INFO:teuthology.kill:2022-09-15 16:43:24,721.721 INFO:teuthology.orchestra.console:Power off smithi169 2022-09-15T16:43:37.203 INFO:teuthology.kill:2022-09-15 16:43:37,202.202 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2022-09-15T16:43:37.209 INFO:teuthology.kill:2022-09-15 16:43:37,209.209 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2022-09-15T16:43:37.361 INFO:teuthology.kill:2022-09-15 16:43:37,360.360 INFO:teuthology.lock.ops:unlocked: smithi096.front.sepia.ceph.com 2022-09-15T16:43:37.367 INFO:teuthology.kill:2022-09-15 16:43:37,367.367 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com 2022-09-15T16:45:37.856 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-15T16:45:38.017 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi096 but it was locked by another job: /home/teuthworker/archive/lflores-2022-09-15_14:32:14-rados:thrash-erasure-code-main-distro-default-smithi/7033987 2022-09-15T16:45:38.017 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi169 but it was locked by another job: /home/teuthworker/archive/lflores-2022-09-15_14:32:14-rados:thrash-erasure-code-main-distro-default-smithi/7033987