2022-09-05T10:04:49.641 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-05T10:04:49.792 INFO:teuthology.lock.ops:Start node 'smithi044.front.sepia.ceph.com' reimaging 2022-09-05T10:04:49.794 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: reset os type and version on server 2022-09-05T10:04:49.795 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2022-09-05T10:04:49.815 INFO:teuthology.lock.ops:Node 'smithi044.front.sepia.ceph.com' reimaging is complete 2022-09-05T10:04:49.815 INFO:teuthology.lock.ops:Start node 'smithi097.front.sepia.ceph.com' reimaging 2022-09-05T10:04:49.816 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: reset os type and version on server 2022-09-05T10:04:49.816 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2022-09-05T10:04:49.836 INFO:teuthology.lock.ops:Node 'smithi097.front.sepia.ceph.com' reimaging is complete 2022-09-05T10:04:50.080 INFO:teuthology.provision.fog.smithi044:Scheduling deploy of rhel 8.6 2022-09-05T10:04:50.130 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of rhel 8.6 2022-09-05T10:04:50.432 INFO:teuthology.orchestra.console:Power off smithi044 2022-09-05T10:04:50.476 INFO:teuthology.orchestra.console:Power off smithi097 2022-09-05T10:04:58.819 INFO:teuthology.orchestra.console:Power off for smithi044 completed 2022-09-05T10:04:58.920 INFO:teuthology.orchestra.console:Power on smithi044 2022-09-05T10:05:02.990 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2022-09-05T10:05:03.092 INFO:teuthology.orchestra.console:Power on smithi097 2022-09-05T10:05:07.292 INFO:teuthology.orchestra.console:Power on for smithi044 completed 2022-09-05T10:05:07.394 INFO:teuthology.provision.fog.smithi044:Waiting for deploy to finish 2022-09-05T10:05:11.482 INFO:teuthology.orchestra.console:Power on for smithi097 completed 2022-09-05T10:05:11.585 INFO:teuthology.provision.fog.smithi097:Waiting for deploy to finish 2022-09-05T10:07:23.583 ERROR:teuthology.orchestra.connection:Error authenticating with smithi044.front.sepia.ceph.com: Authentication failed. 2022-09-05T10:08:58.028 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-05T10:09:13.656 INFO:teuthology.provision.fog.smithi044:Node is ready 2022-09-05T10:09:13.676 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044.front.sepia.ceph.com 2022-09-05T10:09:13.733 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44 smithi044.front.sepia.ceph.com smithi044 2022-09-05T10:09:14.081 INFO:teuthology.provision.fog.smithi044:Deploy complete! 2022-09-05T10:09:58.310 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-05T10:09:58.565 INFO:teuthology.provision.fog.smithi097:Node is ready 2022-09-05T10:09:58.627 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 2022-09-05T10:09:58.689 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97 smithi097.front.sepia.ceph.com smithi097 2022-09-05T10:09:59.024 INFO:teuthology.provision.fog.smithi097:Deploy complete! 2022-09-05T10:09:59.222 INFO:teuthology.lock.ops:Checking smithi044.front.sepia.ceph.com 2022-09-05T10:09:59.223 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-05T10:09:59.247 INFO:teuthology.lock.ops:Checking smithi097.front.sepia.ceph.com 2022-09-05T10:09:59.247 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-05T10:09:59.268 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: set os type and version on server 2022-09-05T10:09:59.269 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2022-09-05T10:09:59.480 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2022-09-05T10:09:59.540 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="Red Hat Enterprise Linux" 2022-09-05T10:09:59.541 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8.6 (Ootpa)" 2022-09-05T10:09:59.541 INFO:teuthology.orchestra.run.smithi044.stdout:ID="rhel" 2022-09-05T10:09:59.541 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="fedora" 2022-09-05T10:09:59.541 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8.6" 2022-09-05T10:09:59.542 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2022-09-05T10:09:59.542 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-05T10:09:59.542 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2022-09-05T10:09:59.542 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-05T10:09:59.543 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.redhat.com/" 2022-09-05T10:09:59.543 INFO:teuthology.orchestra.run.smithi044.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-05T10:09:59.543 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-05T10:09:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: 2022-09-05T10:09:59.544 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-05T10:09:59.544 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-05T10:09:59.544 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-05T10:09:59.544 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-05T10:09:59.546 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2022-09-05T10:09:59.567 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: set os type and version on server 2022-09-05T10:09:59.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2022-09-05T10:09:59.815 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2022-09-05T10:09:59.871 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2022-09-05T10:09:59.872 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2022-09-05T10:09:59.873 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2022-09-05T10:09:59.873 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2022-09-05T10:09:59.873 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2022-09-05T10:09:59.874 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2022-09-05T10:09:59.874 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-05T10:09:59.874 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2022-09-05T10:09:59.875 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-05T10:09:59.875 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2022-09-05T10:09:59.875 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-05T10:09:59.876 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-05T10:09:59.876 INFO:teuthology.orchestra.run.smithi097.stdout: 2022-09-05T10:09:59.876 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-05T10:09:59.876 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-05T10:09:59.877 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-05T10:09:59.877 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-05T10:09:59.877 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2022-09-05T10:09:59.985 INFO:teuthology.dispatcher.supervisor:Running job 7010947 2022-09-05T10:09:59.986 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-09-04_20:55:20-rados-WIP_GBH_snap_mapper_one_wayX1-distro-default-smithi/7010947 --name benhanokh-2022-09-04_20:55:20-rados-WIP_GBH_snap_mapper_one_wayX1-distro-default-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2022-09-04_20:55:20-rados-WIP_GBH_snap_mapper_one_wayX1-distro-default-smithi/7010947/orig.config.yaml 2022-09-05T10:09:59.990 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-09-04_20:55:20-rados-WIP_GBH_snap_mapper_one_wayX1-distro-default-smithi/7010947 2022-09-05T10:09:59.992 INFO:teuthology.dispatcher.supervisor:Job PID: 27649 2022-09-05T10:09:59.992 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-05T22:10:23.065 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-05T22:10:23.231 INFO:teuthology.kill:Killing Pids: {27649} 2022-09-05T22:10:24.859 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-05T22:10:24.860 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-05T22:10:24.861 INFO:teuthology.misc:Compressing logs... 2022-09-05T22:10:24.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2022-09-05T22:10:25.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2022-09-05T22:10:25.649 INFO:teuthology.misc:Compressing logs... 2022-09-05T22:10:26.736 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-05T22:10:26.802 INFO:teuthology.orchestra.run.smithi044.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-05T22:14:07.399 INFO:teuthology.misc:Compressing logs... 2022-09-05T22:14:10.397 INFO:teuthology.kill:No teuthology processes running 2022-09-05T22:14:11.857 INFO:teuthology.kill:Nuking machines: ['smithi044', 'smithi097'] 2022-09-05T22:14:13.112 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-05T22:14:13.113 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-05T22:14:13.114 INFO:teuthology.kill:2022-09-05 22:14:13,112.112 INFO:teuthology.nuke:targets: 2022-09-05T22:14:13.114 INFO:teuthology.kill: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJILye8c3bMqNWKsCNaPel6WvRvsHLKu3NUecMWX7+Bjaed97znpeyFdcMBcMP6W0t1SiZ3E9ZURB2FmO8UL6HA= 2022-09-05T22:14:13.114 INFO:teuthology.kill: smithi097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDt26B3aqr+0MWpvCY8CM2gazY1mZc0yIOkVS9ivhuJw2oIxIXvVq6Skr9UUceTYMpnVB3yxjWTm81OkQbrtzvUbZi5KIdCSaKx48Bx2bw1VW39/EAF5BFnCWvImzhaJHbkULW01BI0yrzAm+W++D12kk5ZGad9Wxt3NaE8XgmcsaJtden2r+WUGI+tIbahRAv9kJMt0YQ8AoOXuxoWrgjwW8dL6RRNxQECI3BnI/spsIFo+RJFpT2Ci8SbEwOT/QYbkQGR9vvoXXF75ZvLTfxIYroQ7jZN74WUB12tmI4q6sR/Zfdi0zt/CSpv31hJPOV1Nr2oC7JPFhYK7cOTqkWqNJelREInHtzcyZ5PbQccRWfqxuKRJBON36tsxgzK9TSdm6wCxoEBbffO4xHbOZGbNHEyB1PzULY8NMabEGXMdOxALJJ1U4H2VL8OpZ7K1KeKR7RBPl0VbmkkYe+lr+ECpiERaP31jg98r1W4JyhjIXrr3ioO5E82l43UI4Hgrp8= 2022-09-05T22:14:13.115 INFO:teuthology.kill:2022-09-05 22:14:13,112.112 INFO:teuthology.nuke:Checking targets against current locks 2022-09-05T22:14:13.172 INFO:teuthology.kill:2022-09-05 22:14:13,172.172 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-05T22:14:13.220 INFO:teuthology.kill:2022-09-05 22:14:13,220.220 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-05T22:14:13.278 INFO:teuthology.kill:2022-09-05 22:14:13,278.278 INFO:teuthology.orchestra.console:Power off smithi044 2022-09-05T22:14:13.323 INFO:teuthology.kill:2022-09-05 22:14:13,322.322 INFO:teuthology.orchestra.console:Power off smithi097 2022-09-05T22:14:21.661 INFO:teuthology.kill:2022-09-05 22:14:21,661.661 INFO:teuthology.orchestra.console:Power off for smithi044 completed 2022-09-05T22:14:21.696 INFO:teuthology.kill:2022-09-05 22:14:21,696.696 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2022-09-05T22:14:21.816 INFO:teuthology.kill:2022-09-05 22:14:21,815.815 INFO:teuthology.lock.ops:unlocked: smithi044.front.sepia.ceph.com 2022-09-05T22:14:21.846 INFO:teuthology.kill:2022-09-05 22:14:21,845.845 INFO:teuthology.lock.ops:unlocked: smithi097.front.sepia.ceph.com 2022-09-05T22:16:22.130 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-05T22:16:22.270 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi044 but it was locked by another job: /home/teuthworker/archive/adking-2022-09-05_21:29:03-orch:cephadm-wip-adk-testing-2022-09-05-1506-distro-default-smithi/7012634 2022-09-05T22:16:22.271 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi097 but it was locked by another job: /home/teuthworker/archive/adking-2022-09-05_21:29:03-orch:cephadm-wip-adk-testing-2022-09-05-1506-distro-default-smithi/7012634