2022-09-14T15:38:06.124 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-14T15:38:06.268 INFO:teuthology.lock.ops:Start node 'smithi008.front.sepia.ceph.com' reimaging 2022-09-14T15:38:06.269 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: reset os type and version on server 2022-09-14T15:38:06.269 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2022-09-14T15:38:06.291 INFO:teuthology.lock.ops:Node 'smithi008.front.sepia.ceph.com' reimaging is complete 2022-09-14T15:38:06.292 INFO:teuthology.lock.ops:Start node 'smithi052.front.sepia.ceph.com' reimaging 2022-09-14T15:38:06.292 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: reset os type and version on server 2022-09-14T15:38:06.293 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2022-09-14T15:38:06.315 INFO:teuthology.lock.ops:Node 'smithi052.front.sepia.ceph.com' reimaging is complete 2022-09-14T15:38:06.572 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of rhel 8.4 2022-09-14T15:38:06.588 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of rhel 8.4 2022-09-14T15:38:06.943 INFO:teuthology.orchestra.console:Power off smithi052 2022-09-14T15:38:06.989 INFO:teuthology.orchestra.console:Power off smithi008 2022-09-14T15:38:15.350 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2022-09-14T15:38:15.452 INFO:teuthology.orchestra.console:Power on smithi052 2022-09-14T15:38:19.487 INFO:teuthology.orchestra.console:Power off for smithi008 completed 2022-09-14T15:38:19.589 INFO:teuthology.orchestra.console:Power on smithi008 2022-09-14T15:38:23.848 INFO:teuthology.orchestra.console:Power on for smithi008 completed 2022-09-14T15:38:23.894 INFO:teuthology.orchestra.console:Power on for smithi052 completed 2022-09-14T15:38:23.950 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish 2022-09-14T15:38:23.995 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish 2022-09-14T15:42:06.219 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-14T15:42:21.149 INFO:teuthology.provision.fog.smithi052:Node is ready 2022-09-14T15:42:21.170 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052.front.sepia.ceph.com 2022-09-14T15:42:21.229 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52 smithi052.front.sepia.ceph.com smithi052 2022-09-14T15:42:21.600 INFO:teuthology.provision.fog.smithi052:Deploy complete! 2022-09-14T15:42:55.850 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-14T15:42:56.062 INFO:teuthology.provision.fog.smithi008:Node is ready 2022-09-14T15:42:56.117 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com 2022-09-14T15:42:56.173 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008 2022-09-14T15:42:56.496 INFO:teuthology.provision.fog.smithi008:Deploy complete! 2022-09-14T15:42:56.703 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com 2022-09-14T15:42:56.704 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-14T15:42:56.728 INFO:teuthology.lock.ops:Checking smithi052.front.sepia.ceph.com 2022-09-14T15:42:56.729 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-14T15:42:56.749 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server 2022-09-14T15:42:56.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2022-09-14T15:42:57.036 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2022-09-14T15:42:57.091 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="Red Hat Enterprise Linux" 2022-09-14T15:42:57.091 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8.4 (Ootpa)" 2022-09-14T15:42:57.091 INFO:teuthology.orchestra.run.smithi008.stdout:ID="rhel" 2022-09-14T15:42:57.092 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="fedora" 2022-09-14T15:42:57.092 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8.4" 2022-09-14T15:42:57.092 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2022-09-14T15:42:57.093 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-09-14T15:42:57.093 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2022-09-14T15:42:57.093 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-09-14T15:42:57.093 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.redhat.com/" 2022-09-14T15:42:57.094 INFO:teuthology.orchestra.run.smithi008.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-14T15:42:57.094 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-14T15:42:57.094 INFO:teuthology.orchestra.run.smithi008.stdout: 2022-09-14T15:42:57.094 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-14T15:42:57.095 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-09-14T15:42:57.095 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-14T15:42:57.095 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-09-14T15:42:57.097 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2022-09-14T15:42:57.117 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: set os type and version on server 2022-09-14T15:42:57.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2022-09-14T15:42:57.371 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2022-09-14T15:42:57.428 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="Red Hat Enterprise Linux" 2022-09-14T15:42:57.428 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="8.4 (Ootpa)" 2022-09-14T15:42:57.428 INFO:teuthology.orchestra.run.smithi052.stdout:ID="rhel" 2022-09-14T15:42:57.428 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="fedora" 2022-09-14T15:42:57.429 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="8.4" 2022-09-14T15:42:57.429 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el8" 2022-09-14T15:42:57.429 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-09-14T15:42:57.429 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2022-09-14T15:42:57.430 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-09-14T15:42:57.430 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://www.redhat.com/" 2022-09-14T15:42:57.430 INFO:teuthology.orchestra.run.smithi052.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-14T15:42:57.431 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-14T15:42:57.431 INFO:teuthology.orchestra.run.smithi052.stdout: 2022-09-14T15:42:57.433 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-14T15:42:57.434 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-09-14T15:42:57.434 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-14T15:42:57.434 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-09-14T15:42:57.435 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2022-09-14T15:42:57.558 INFO:teuthology.dispatcher.supervisor:Running job 7032603 2022-09-14T15:42:57.559 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-09-14_15:11:39-rados-quincy-release-distro-default-smithi/7032603 --name lflores-2022-09-14_15:11:39-rados-quincy-release-distro-default-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} -- /home/teuthworker/archive/lflores-2022-09-14_15:11:39-rados-quincy-release-distro-default-smithi/7032603/orig.config.yaml 2022-09-14T15:42:57.564 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-09-14_15:11:39-rados-quincy-release-distro-default-smithi/7032603 2022-09-14T15:42:57.565 INFO:teuthology.dispatcher.supervisor:Job PID: 29058 2022-09-14T15:42:57.566 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-15T03:43:56.218 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-15T03:43:56.403 INFO:teuthology.kill:Killing Pids: {29058} 2022-09-15T03:43:57.945 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-15T03:43:57.946 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-15T03:43:57.946 INFO:teuthology.misc:Compressing logs... 2022-09-15T03:43:57.947 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2022-09-15T03:43:58.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2022-09-15T03:43:58.525 INFO:teuthology.misc:Compressing logs... 2022-09-15T03:43:59.685 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-15T03:43:59.691 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-15T03:44:00.342 INFO:teuthology.misc:Compressing logs... 2022-09-15T03:51:14.547 INFO:teuthology.kill:No teuthology processes running 2022-09-15T03:51:15.870 INFO:teuthology.kill:Nuking machines: ['smithi008', 'smithi052'] 2022-09-15T03:51:17.056 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-15T03:51:17.057 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-15T03:51:17.058 INFO:teuthology.kill:2022-09-15 03:51:17,055.055 INFO:teuthology.nuke:targets: 2022-09-15T03:51:17.058 INFO:teuthology.kill: smithi008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC751Lt3nxgPdtuPdyRGM+KqjyKqDEI9Bj7DGq+HGVrt/HPOFLbNprZzn/YR2qmzE2AtbjYcKAUBI0/BEGuMYD3eHIcDO+tMRLdztjGK1qLYQwRHbY9xXeEUeKsSnUqAm6EF5LWsScboLfBiDs+eiugpnAGYpj+LVdJ6WXRrMhMgx/09bEpLCfLVAvezUeNtUaLTfFKXGxbbGGc7krLNpr4OWUD5VqegQOtoejgb0di3pqByTKm7oF9ExD7OHBQIqW8LDFjS+J1l3u3B48yhhM7JrUr3LCSE1NKKrgQbiP1aSiRnzTmt9cP/1inQ/gpLtWLc4KzmDdKjP6Y7ShOruvGFevqzl9e+JU92Vb2nyju+bU7vCVsgSi/TDCLQrzOdWmPF5GiuOi7+h+ZHkFsQ/FXO/Xv6Lme8gzL7/HMaUlof6yLt+uzXIsMsMDKi8IE2vA2+S7oO5+xW1s0mLOyKa3cABlf2ICZ/zRucQrKp1wCJrCvBd8pCkPepu5Quk2anWs= 2022-09-15T03:51:17.059 INFO:teuthology.kill: smithi052.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDww8SrbNg+kCacYHS+/uGbPWD3FLFASeV6OFnByAgt3n85+KgIQGaxCG1S7rylg5k+zqIHn+mOfPs/bhfNiTDwfAgGwmv179um4PyoMiHPu5Ch4xTc/RAF2g/nR5R6QMzP6bdSz7nHyuYYJkb1sObJUC0b169wqd/oqKy9hVjy4LVe1R2INg2dS5TztAAQbCia2z58VEd2H4FT1wV6in/QCJHDCsFeqGrEmTQtdWgdG36qQ5XgBVFUA8jNd2Oyl7ZRBsRcsmSwt7QJL88xMkMmO/znmECe2topLcERwc5Hqaj2ip07H2hRUQ8invhtfcIm2iWlJvUSmZVoFn9Yn3ftwDN3i/WNTvK23+xvdTyY35ic/4k83jjRItA0vFhp45+uynOGLaXykh+HjARJrZqsQowgZ5a2F0ksKJJAL13fn5UeQ6dOWrQ4KqsWru2cRoIgk14lTPiRwbRNYb3LvhBvEOuX04ZJ9+fdk4IOEdEKvOlIBglSDgjYGGCm81MC5E= 2022-09-15T03:51:17.059 INFO:teuthology.kill:2022-09-15 03:51:17,055.055 INFO:teuthology.nuke:Checking targets against current locks 2022-09-15T03:51:17.113 INFO:teuthology.kill:2022-09-15 03:51:17,113.113 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-15T03:51:17.144 INFO:teuthology.kill:2022-09-15 03:51:17,144.144 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-15T03:51:17.208 INFO:teuthology.kill:2022-09-15 03:51:17,207.207 INFO:teuthology.orchestra.console:Power off smithi008 2022-09-15T03:51:17.228 INFO:teuthology.kill:2022-09-15 03:51:17,227.227 INFO:teuthology.orchestra.console:Power off smithi052 2022-09-15T03:51:25.595 INFO:teuthology.kill:2022-09-15 03:51:25,595.595 INFO:teuthology.orchestra.console:Power off for smithi008 completed 2022-09-15T03:51:25.601 INFO:teuthology.kill:2022-09-15 03:51:25,600.600 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2022-09-15T03:51:25.751 INFO:teuthology.kill:2022-09-15 03:51:25,750.750 INFO:teuthology.lock.ops:unlocked: smithi008.front.sepia.ceph.com 2022-09-15T03:51:25.753 INFO:teuthology.kill:2022-09-15 03:51:25,753.753 INFO:teuthology.lock.ops:unlocked: smithi052.front.sepia.ceph.com 2022-09-15T03:53:26.110 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-15T03:53:26.227 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi052 but it was locked by another job: /home/teuthworker/archive/rishabh-2022-09-14_12:48:43-fs-wip-rishabh-testing-2022Sep9-1708-testing-default-smithi/7032061