2022-09-04T14:52:33.050 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-04T14:52:33.201 INFO:teuthology.lock.ops:Start node 'smithi059.front.sepia.ceph.com' reimaging 2022-09-04T14:52:33.202 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: reset os type and version on server 2022-09-04T14:52:33.202 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-09-04T14:52:33.226 INFO:teuthology.lock.ops:Node 'smithi059.front.sepia.ceph.com' reimaging is complete 2022-09-04T14:52:33.227 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging 2022-09-04T14:52:33.227 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server 2022-09-04T14:52:33.228 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-09-04T14:52:33.253 INFO:teuthology.lock.ops:Node 'smithi121.front.sepia.ceph.com' reimaging is complete 2022-09-04T14:52:33.445 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of rhel 8.6 2022-09-04T14:52:33.470 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of rhel 8.6 2022-09-04T14:52:33.798 INFO:teuthology.orchestra.console:Power off smithi059 2022-09-04T14:52:33.849 INFO:teuthology.orchestra.console:Power off smithi121 2022-09-04T14:52:33.942 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-09-04T14:52:33.991 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-09-04T14:52:34.044 INFO:teuthology.orchestra.console:Power on smithi059 2022-09-04T14:52:34.092 INFO:teuthology.orchestra.console:Power on smithi121 2022-09-04T14:52:38.299 INFO:teuthology.orchestra.console:Power on for smithi059 completed 2022-09-04T14:52:38.400 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2022-09-04T14:52:42.453 INFO:teuthology.orchestra.console:Power on for smithi121 completed 2022-09-04T14:52:42.555 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2022-09-04T14:55:13.626 ERROR:teuthology.orchestra.connection:Error authenticating with smithi121.front.sepia.ceph.com: Authentication failed. 2022-09-04T14:56:51.949 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-04T14:56:52.199 INFO:teuthology.provision.fog.smithi059:Node is ready 2022-09-04T14:56:52.218 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2022-09-04T14:56:52.329 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2022-09-04T14:56:52.550 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-04T14:56:52.695 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2022-09-04T14:57:17.522 INFO:teuthology.provision.fog.smithi121:Node is ready 2022-09-04T14:57:17.545 INFO:teuthology.orchestra.run.smithi121.stdout:smithi055 2022-09-04T14:57:17.755 INFO:teuthology.provision.fog.smithi121:Deploy complete! 2022-09-04T14:57:17.964 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2022-09-04T14:57:17.965 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-04T14:57:17.985 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com 2022-09-04T14:57:17.985 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-04T14:57:18.005 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2022-09-04T14:57:18.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-09-04T14:57:18.275 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2022-09-04T14:57:18.333 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2022-09-04T14:57:18.333 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.6 (Ootpa)" 2022-09-04T14:57:18.333 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2022-09-04T14:57:18.334 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2022-09-04T14:57:18.334 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.6" 2022-09-04T14:57:18.334 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2022-09-04T14:57:18.334 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-04T14:57:18.335 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2022-09-04T14:57:18.335 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-04T14:57:18.335 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2022-09-04T14:57:18.336 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-04T14:57:18.336 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-04T14:57:18.336 INFO:teuthology.orchestra.run.smithi059.stdout: 2022-09-04T14:57:18.336 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-04T14:57:18.336 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-04T14:57:18.337 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-04T14:57:18.337 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-04T14:57:18.338 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-09-04T14:57:18.358 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server 2022-09-04T14:57:18.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-09-04T14:57:18.560 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2022-09-04T14:57:18.616 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux" 2022-09-04T14:57:18.618 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.6 (Ootpa)" 2022-09-04T14:57:18.618 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel" 2022-09-04T14:57:18.619 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora" 2022-09-04T14:57:18.619 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.6" 2022-09-04T14:57:18.619 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2022-09-04T14:57:18.619 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-04T14:57:18.619 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2022-09-04T14:57:18.620 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-04T14:57:18.620 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/" 2022-09-04T14:57:18.620 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-04T14:57:18.620 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-04T14:57:18.620 INFO:teuthology.orchestra.run.smithi121.stdout: 2022-09-04T14:57:18.621 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-04T14:57:18.621 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-04T14:57:18.621 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-04T14:57:18.621 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-04T14:57:18.622 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-09-04T14:57:18.737 INFO:teuthology.dispatcher.supervisor:Running job 7004696 2022-09-04T14:57:18.738 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004696 --name benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004696/orig.config.yaml 2022-09-04T14:57:18.742 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004696 2022-09-04T14:57:18.743 INFO:teuthology.dispatcher.supervisor:Job PID: 23161 2022-09-04T14:57:18.743 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-05T02:57:42.426 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-05T02:57:42.637 INFO:teuthology.kill:Killing Pids: {23161} 2022-09-05T02:57:44.060 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-05T02:57:44.061 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-05T02:57:44.062 INFO:teuthology.misc:Compressing logs... 2022-09-05T02:57:44.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-09-05T02:57:44.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-09-05T02:57:44.603 INFO:teuthology.misc:Compressing logs... 2022-09-05T02:57:45.544 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-05T02:57:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-05T02:57:59.267 INFO:teuthology.misc:Compressing logs... 2022-09-05T02:58:00.696 INFO:teuthology.kill:No teuthology processes running 2022-09-05T02:58:01.985 INFO:teuthology.kill:Nuking machines: ['smithi059', 'smithi121'] 2022-09-05T02:58:03.101 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-05T02:58:03.102 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-05T02:58:03.103 INFO:teuthology.kill:2022-09-05 02:58:03,101.101 INFO:teuthology.nuke:targets: 2022-09-05T02:58:03.103 INFO:teuthology.kill: smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4SwVKwAjFFqrQIe5zsQMmcsTeC9fMAL/bMsQrYCA0+efCFRDs+G2zD4WbEoyGcfY7B9kBKL1Yv3xNxoHCXDDhP9ou9ILbSozFz64vq2+aTIkhZKyPqE1QUQ4e2Qv/k3J+VP5XpwMIIMDGfV1fOVsQnJurd00fGtwOp0QTbpJYV8wggdctAH6/zsQb6I9kkkm+1ZHSz2VRFp7o4YHYEDd4RCjv0IOiOvfSrejfvcoa0X4ylbHkIctubb3+O2Jcag9Y3nHk/E0RRHKlwih9tgEXoZCphvgErZMG8w8i9iH7oqbqnBprXETWqYEixFZWCIq2QypcZciKfJCK/gLU84xi6zBgIMo32ulNFvB59l6KxsqRWc7OXql/mijqOGXH/ArBsQa7XHlCxZlZNymz79pJZYD1ny3A4wY+elSL0bqEYghtRjK5xisxc7j3FiQNJh8ifOILcvv6PE7LU3j3osAbbIC3L273JoAVYrlu5v+GwfKtWn3qtBjCmCd7yCDFa58= 2022-09-05T02:58:03.103 INFO:teuthology.kill: smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCY9zYjmBwHM9OnHrfzWm7MYZtQziV7L3ZmMRz5HJPocMOV5pAPCK7EriKKSIK/xs2qd22plOo9+cdNwPvCjOcBcnft+L+crBKU2aNp8oan1lapiz4TIbM62q7D7LDPRntRfplMCbi8WkeucwxZzWom8xJ/dIZ81e7yEKnK53T8dj8B7zQm8QDtvv9+tGWeFqRTxz5asnxs0Ii+3XRFzXVo8x6CqdtC7DKygt5rIgpiM/HYnX5fcVthGcLCr5tNb5AJAhvbiPZay5hxzfJiy2C1BnO5t+PuRCshRmlzqZxwk3fWqtGjs/C9Nvf5N4TczDrdKF23XK2hqRnYRtvnwClnBL1BH3/leXbccbJa04RqlUPdMUU40FQWFEBAtJ1ejLFlHkeHlWXkPujtN+kNPUF2araHPnA9uGCzeMAMVK5SvIUYxvWKhtvUgCSiPNRhKLtGGc6OhWthmoGJYlplGi9qiP7BKHNkM8SdgpfC++svvkbzyKkhdl58RXUt0jY1Rr0= 2022-09-05T02:58:03.104 INFO:teuthology.kill:2022-09-05 02:58:03,101.101 INFO:teuthology.nuke:Checking targets against current locks 2022-09-05T02:58:03.155 INFO:teuthology.kill:2022-09-05 02:58:03,155.155 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-05T02:58:03.179 INFO:teuthology.kill:2022-09-05 02:58:03,178.178 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-05T02:58:03.235 INFO:teuthology.kill:2022-09-05 02:58:03,234.234 INFO:teuthology.orchestra.console:Power off smithi059 2022-09-05T02:58:03.269 INFO:teuthology.kill:2022-09-05 02:58:03,269.269 INFO:teuthology.orchestra.console:Power off smithi121 2022-09-05T02:58:11.602 INFO:teuthology.kill:2022-09-05 02:58:11,602.602 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-09-05T02:58:11.752 INFO:teuthology.kill:2022-09-05 02:58:11,751.751 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2022-09-05T02:58:15.734 INFO:teuthology.kill:2022-09-05 02:58:15,733.733 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-09-05T02:58:15.882 INFO:teuthology.kill:2022-09-05 02:58:15,881.881 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com 2022-09-05T03:00:16.143 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-05T03:00:16.264 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi059 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-04_23:15:18-rbd-wip-yuri10-testing-2022-09-04-0811-quincy-distro-default-smithi/7011410 2022-09-05T03:00:16.264 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi121 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-04_23:15:18-rbd-wip-yuri10-testing-2022-09-04-0811-quincy-distro-default-smithi/7011410