2022-09-01T12:18:39.762 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-01T12:18:39.910 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging 2022-09-01T12:18:39.911 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server 2022-09-01T12:18:39.912 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-09-01T12:18:39.933 INFO:teuthology.lock.ops:Node 'smithi002.front.sepia.ceph.com' reimaging is complete 2022-09-01T12:18:39.934 INFO:teuthology.lock.ops:Start node 'smithi038.front.sepia.ceph.com' reimaging 2022-09-01T12:18:39.934 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: reset os type and version on server 2022-09-01T12:18:39.934 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2022-09-01T12:18:39.956 INFO:teuthology.lock.ops:Node 'smithi038.front.sepia.ceph.com' reimaging is complete 2022-09-01T12:18:40.152 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of rhel 8.6 2022-09-01T12:18:40.204 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of rhel 8.6 2022-09-01T12:18:40.503 INFO:teuthology.orchestra.console:Power off smithi002 2022-09-01T12:18:40.557 INFO:teuthology.orchestra.console:Power off smithi038 2022-09-01T12:18:40.703 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2022-09-01T12:18:40.804 INFO:teuthology.orchestra.console:Power on smithi038 2022-09-01T12:18:48.896 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2022-09-01T12:18:48.999 INFO:teuthology.orchestra.console:Power on smithi002 2022-09-01T12:18:49.173 INFO:teuthology.orchestra.console:Power on for smithi038 completed 2022-09-01T12:18:49.274 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish 2022-09-01T12:18:57.371 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2022-09-01T12:18:57.473 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2022-09-01T12:22:39.333 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-01T12:22:59.637 INFO:teuthology.provision.fog.smithi002:Node is ready 2022-09-01T12:22:59.657 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2022-09-01T12:22:59.714 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2022-09-01T12:23:00.076 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2022-09-01T12:23:05.801 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-01T12:23:06.071 INFO:teuthology.provision.fog.smithi038:Node is ready 2022-09-01T12:23:06.095 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com 2022-09-01T12:23:06.156 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038 2022-09-01T12:23:06.541 INFO:teuthology.provision.fog.smithi038:Deploy complete! 2022-09-01T12:23:06.830 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2022-09-01T12:23:06.832 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-01T12:23:06.854 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com 2022-09-01T12:23:06.854 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-01T12:23:06.880 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2022-09-01T12:23:06.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-09-01T12:23:07.120 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2022-09-01T12:23:07.180 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Red Hat Enterprise Linux" 2022-09-01T12:23:07.181 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8.6 (Ootpa)" 2022-09-01T12:23:07.181 INFO:teuthology.orchestra.run.smithi002.stdout:ID="rhel" 2022-09-01T12:23:07.181 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="fedora" 2022-09-01T12:23:07.182 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8.6" 2022-09-01T12:23:07.182 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2022-09-01T12:23:07.182 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-01T12:23:07.183 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2022-09-01T12:23:07.183 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-01T12:23:07.183 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.redhat.com/" 2022-09-01T12:23:07.183 INFO:teuthology.orchestra.run.smithi002.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-01T12:23:07.183 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-01T12:23:07.184 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-09-01T12:23:07.184 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-01T12:23:07.184 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-01T12:23:07.184 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-01T12:23:07.185 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-01T12:23:07.186 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-09-01T12:23:07.212 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server 2022-09-01T12:23:07.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2022-09-01T12:23:07.421 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2022-09-01T12:23:07.480 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Red Hat Enterprise Linux" 2022-09-01T12:23:07.481 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8.6 (Ootpa)" 2022-09-01T12:23:07.482 INFO:teuthology.orchestra.run.smithi038.stdout:ID="rhel" 2022-09-01T12:23:07.482 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="fedora" 2022-09-01T12:23:07.483 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8.6" 2022-09-01T12:23:07.483 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8" 2022-09-01T12:23:07.484 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-01T12:23:07.484 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2022-09-01T12:23:07.485 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-01T12:23:07.485 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.redhat.com/" 2022-09-01T12:23:07.486 INFO:teuthology.orchestra.run.smithi038.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-01T12:23:07.486 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-01T12:23:07.487 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-09-01T12:23:07.487 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-01T12:23:07.488 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-01T12:23:07.488 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-01T12:23:07.489 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-01T12:23:07.490 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2022-09-01T12:23:07.596 INFO:teuthology.dispatcher.supervisor:Running job 7004160 2022-09-01T12:23:07.597 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-09-01_10:32:47-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004160 --name benhanokh-2022-09-01_10:32:47-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} -- /home/teuthworker/archive/benhanokh-2022-09-01_10:32:47-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004160/orig.config.yaml 2022-09-01T12:23:07.603 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-09-01_10:32:47-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004160 2022-09-01T12:23:07.604 INFO:teuthology.dispatcher.supervisor:Job PID: 24368 2022-09-01T12:23:07.604 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-02T00:24:07.811 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-02T00:24:08.113 INFO:teuthology.kill:Killing Pids: {24368} 2022-09-02T00:24:10.542 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-02T00:24:10.543 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-02T00:24:10.543 INFO:teuthology.misc:Compressing logs... 2022-09-02T00:24:10.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-09-02T00:24:10.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2022-09-02T00:24:11.484 INFO:teuthology.misc:Compressing logs... 2022-09-02T00:24:12.604 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-02T00:24:12.625 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-02T00:26:47.018 INFO:teuthology.misc:Compressing logs... 2022-09-02T00:26:52.373 INFO:teuthology.kill:No teuthology processes running 2022-09-02T00:26:53.997 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi038'] 2022-09-02T00:26:55.406 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-02T00:26:55.407 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-02T00:26:55.407 INFO:teuthology.kill:2022-09-02 00:26:55,405.405 INFO:teuthology.nuke:targets: 2022-09-02T00:26:55.408 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/p0ud06r+xbTNi9wKMrWC8Xe7WpWktQWg5GA0pENo9WjRG9SEKXDBWjKEhCJs8HCR1PrXrfzWJZc0MfJ0XJEMOxaRWLEnKj62iUeDG2VTCGPX9xERH9g/KgO1tjz9IWVIDMk4swtcT8DooFmDyYrmpfeote9jCn3F7J+kR2sqZb4ouu9d5xUvZWDBxTh2igs863vyWxDEoe93Y+yFDNk4IPjxPV0cv8e4nthjqIekcc9rQ2nf2uswu5vHOXoOs+CVmudNFrnHL3q8vROED8ucCgIkPKbtHGCiPwANOPPCVbiK7OE+Si9zZeQR8sPWXIIghKIvOypaksr6YJsG/MjW4+w0hcLmygz1c72iC4eWgs2S+Qx6+ZFKnoA6ecAhdQ0xSGPKsMlkSdPegnaRI9W/UE31e9h6xC48ATJ3y/2vVTO9qjiEGXgc6mMcnrRrFF10/805XLRx/3MZm6a4sT6d9NhzGToDDqu5C5YE7DAQSdkpsHqapbfa8vWTVCWByK8= 2022-09-02T00:26:55.408 INFO:teuthology.kill: smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC49v72U8GO/rzMaAx70EaMsBiBEHmtBIf8H6wszHfoOVrXOGmxKRi4eQjbmmacbqmivuWnXomslAvbyC0fCidtEyTY4LVbIZ2XnbfTry2FNWgaV/vwu9sbb93wL9e9U7J8vsQ8xtR876481NobjT/XrmvEPWhn9jWl4pIwAZQzQDleUbFbo9qwsPBMXGzoZekIuE/CWdJ0R99sCC2erLRO3OD3yoH2a5mEV0ibdNSjMKOq/6AxpDkEdnk6BI9j6bex34iIgKgMxrI9T8i+Gyaeit/ws5P0hr6x52kS3qw2cbg794DvL8+75olsuRHARE4cXNSNdvQDeuxcDtPmLGmLzKmaRgYByI78+kSTGzDUSVjoeQeFqytoOf+BmgSI5roCcrvgpIj2Of5U+OOvwI+uR6CzguC0aVhcFbdy8JD4Js1GPbqiFvbikgQJsQJIiM05+yWp16Zv5P/OiMh028f1tLMvZ80tbzFO8LjX579eXqMdZNIS/Wwif8eZuWJelN0= 2022-09-02T00:26:55.409 INFO:teuthology.kill:2022-09-02 00:26:55,408.408 INFO:teuthology.nuke:Checking targets against current locks 2022-09-02T00:26:55.470 INFO:teuthology.kill:2022-09-02 00:26:55,468.468 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-02T00:26:55.519 INFO:teuthology.kill:2022-09-02 00:26:55,518.518 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-02T00:26:55.564 INFO:teuthology.kill:2022-09-02 00:26:55,563.563 INFO:teuthology.orchestra.console:Power off smithi002 2022-09-02T00:26:55.678 INFO:teuthology.kill:2022-09-02 00:26:55,677.677 INFO:teuthology.orchestra.console:Power off smithi038 2022-09-02T00:27:03.990 INFO:teuthology.kill:2022-09-02 00:27:03,989.989 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2022-09-02T00:27:04.158 INFO:teuthology.kill:2022-09-02 00:27:04,157.157 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com 2022-09-02T00:27:08.182 INFO:teuthology.kill:2022-09-02 00:27:08,181.181 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2022-09-02T00:27:08.338 INFO:teuthology.kill:2022-09-02 00:27:08,338.338 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com 2022-09-02T00:29:08.757 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-02T00:29:08.901 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi002 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-01_20:33:38-rados-wip-yuri6-testing-2022-09-01-1034-quincy-distro-default-smithi/7005979 2022-09-02T00:29:08.902 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi038 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-01_20:33:38-rados-wip-yuri6-testing-2022-09-01-1034-quincy-distro-default-smithi/7005978