2022-09-04T14:16:05.494 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-04T14:16:05.693 INFO:teuthology.lock.ops:Start node 'smithi046.front.sepia.ceph.com' reimaging 2022-09-04T14:16:05.694 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: reset os type and version on server 2022-09-04T14:16:05.694 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2022-09-04T14:16:05.721 INFO:teuthology.lock.ops:Node 'smithi046.front.sepia.ceph.com' reimaging is complete 2022-09-04T14:16:05.722 INFO:teuthology.lock.ops:Start node 'smithi116.front.sepia.ceph.com' reimaging 2022-09-04T14:16:05.722 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: reset os type and version on server 2022-09-04T14:16:05.723 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2022-09-04T14:16:05.752 INFO:teuthology.lock.ops:Node 'smithi116.front.sepia.ceph.com' reimaging is complete 2022-09-04T14:16:05.752 INFO:teuthology.lock.ops:Start node 'smithi172.front.sepia.ceph.com' reimaging 2022-09-04T14:16:05.753 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: reset os type and version on server 2022-09-04T14:16:05.753 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2022-09-04T14:16:05.775 INFO:teuthology.lock.ops:Node 'smithi172.front.sepia.ceph.com' reimaging is complete 2022-09-04T14:16:05.953 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of centos 8.stream 2022-09-04T14:16:05.966 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of centos 8.stream 2022-09-04T14:16:05.991 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 8.stream 2022-09-04T14:16:06.268 INFO:teuthology.orchestra.console:Power off smithi046 2022-09-04T14:16:06.286 INFO:teuthology.orchestra.console:Power off smithi116 2022-09-04T14:16:06.329 INFO:teuthology.orchestra.console:Power off smithi172 2022-09-04T14:16:06.438 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2022-09-04T14:16:06.472 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2022-09-04T14:16:06.540 INFO:teuthology.orchestra.console:Power on smithi046 2022-09-04T14:16:06.575 INFO:teuthology.orchestra.console:Power on smithi172 2022-09-04T14:16:10.837 INFO:teuthology.orchestra.console:Power on for smithi172 completed 2022-09-04T14:16:10.938 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2022-09-04T14:16:14.664 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2022-09-04T14:16:14.766 INFO:teuthology.orchestra.console:Power on smithi116 2022-09-04T14:16:14.912 INFO:teuthology.orchestra.console:Power on for smithi046 completed 2022-09-04T14:16:15.014 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish 2022-09-04T14:16:23.203 INFO:teuthology.orchestra.console:Power on for smithi116 completed 2022-09-04T14:16:23.304 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2022-09-04T14:20:46.662 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-04T14:20:46.903 INFO:teuthology.provision.fog.smithi046:Node is ready 2022-09-04T14:20:46.922 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com 2022-09-04T14:20:46.982 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2022-09-04T14:20:47.342 INFO:teuthology.provision.fog.smithi046:Deploy complete! 2022-09-04T14:20:54.902 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-04T14:20:55.149 INFO:teuthology.provision.fog.smithi116:Node is ready 2022-09-04T14:20:55.169 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2022-09-04T14:20:55.227 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2022-09-04T14:20:55.574 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2022-09-04T14:20:57.551 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-04T14:20:57.800 INFO:teuthology.provision.fog.smithi172:Node is ready 2022-09-04T14:20:57.819 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com 2022-09-04T14:20:57.877 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2022-09-04T14:20:58.217 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2022-09-04T14:20:58.452 INFO:teuthology.lock.ops:Checking smithi046.front.sepia.ceph.com 2022-09-04T14:20:58.453 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-04T14:20:58.472 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2022-09-04T14:20:58.473 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-04T14:20:58.490 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2022-09-04T14:20:58.491 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-04T14:20:58.512 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: set os type and version on server 2022-09-04T14:20:58.513 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2022-09-04T14:20:58.797 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2022-09-04T14:20:58.856 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2022-09-04T14:20:58.857 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="8" 2022-09-04T14:20:58.857 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2022-09-04T14:20:58.857 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2022-09-04T14:20:58.857 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="8" 2022-09-04T14:20:58.858 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el8" 2022-09-04T14:20:58.858 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-04T14:20:58.858 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2022-09-04T14:20:58.858 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-04T14:20:58.859 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2022-09-04T14:20:58.859 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-04T14:20:58.859 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-04T14:20:58.859 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-04T14:20:58.861 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2022-09-04T14:20:58.883 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2022-09-04T14:20:58.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2022-09-04T14:20:59.092 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2022-09-04T14:20:59.150 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2022-09-04T14:20:59.151 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8" 2022-09-04T14:20:59.151 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2022-09-04T14:20:59.151 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2022-09-04T14:20:59.152 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8" 2022-09-04T14:20:59.152 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8" 2022-09-04T14:20:59.152 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-04T14:20:59.152 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2022-09-04T14:20:59.152 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-04T14:20:59.153 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2022-09-04T14:20:59.153 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-04T14:20:59.153 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-04T14:20:59.154 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-04T14:20:59.154 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2022-09-04T14:20:59.175 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2022-09-04T14:20:59.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2022-09-04T14:20:59.368 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2022-09-04T14:20:59.425 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2022-09-04T14:20:59.426 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8" 2022-09-04T14:20:59.426 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2022-09-04T14:20:59.426 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2022-09-04T14:20:59.426 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8" 2022-09-04T14:20:59.427 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2022-09-04T14:20:59.427 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-04T14:20:59.427 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2022-09-04T14:20:59.427 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-04T14:20:59.427 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2022-09-04T14:20:59.428 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-04T14:20:59.428 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-04T14:20:59.428 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-04T14:20:59.429 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2022-09-04T14:20:59.529 INFO:teuthology.dispatcher.supervisor:Running job 7004663 2022-09-04T14:20:59.530 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004663 --name benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004663/orig.config.yaml 2022-09-04T14:20:59.535 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004663 2022-09-04T14:20:59.535 INFO:teuthology.dispatcher.supervisor:Job PID: 18676 2022-09-04T14:20:59.536 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-05T02:21:57.848 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-05T02:21:58.032 INFO:teuthology.kill:Killing Pids: {18676} 2022-09-05T02:21:59.461 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2022-09-05T02:21:59.462 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2022-09-05T02:21:59.463 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2022-09-05T02:21:59.463 INFO:teuthology.misc:Compressing logs... 2022-09-05T02:21:59.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2022-09-05T02:21:59.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2022-09-05T02:21:59.867 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2022-09-05T02:22:00.454 INFO:teuthology.misc:Compressing logs... 2022-09-05T02:22:01.099 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-05T02:22:01.291 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-05T02:22:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-05T02:25:13.687 INFO:teuthology.misc:Compressing logs... 2022-09-05T02:25:17.552 INFO:teuthology.kill:No teuthology processes running 2022-09-05T02:25:18.964 INFO:teuthology.kill:Nuking machines: ['smithi046', 'smithi116', 'smithi172'] 2022-09-05T02:25:20.081 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-05T02:25:20.082 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-05T02:25:20.082 INFO:teuthology.kill:2022-09-05 02:25:20,081.081 INFO:teuthology.nuke:targets: 2022-09-05T02:25:20.083 INFO:teuthology.kill: smithi046.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCfgskF3Sz0psXHqQkH8nQ1NHsAbVWADpO8QF1r77acdHsmif9TBxWjQW4h7hfhGchq5zbgCy+xk/x56NCr7alLvmO/z8VMO8100m2OTjW0OAgueQncuTDEDiky+2xOKFbTHchJ6x48IXGUJfUV0Ximubk6gr4zv0glCMeLZj2oIMHqsUUKh0NSkC9WQjMcG1AwxtccGGU2xYlhJLivrHcCu5E1RQSThMvo3IOlfj0xWC5ilYB2GDHZ6D5SL3RmzQfXkBzzPcff9EqqhRfWO9sMgAx52CLNb5of8DbYnpNIjsyuKZFhHlUZ74WKWjB3eT3WylGbEctRWyf413Xcv7whZwKQdHRFxrvyK8S1DQNiQduqKTPAMpOiWybdLxiRd43S+k0FHQF4FsadIsthLKFo9MJPOI8m2iyUcYQ9DL4nYyoGeU9UST0dFeQ32yiXwok33k69Y4NaqZI+pl/EoWJMStCt/K5GYC5PDRCIZao7K94Cy2pFFVF6v/PD/iqSf8= 2022-09-05T02:25:20.083 INFO:teuthology.kill: smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR8ENaw6ZV3MuEflAgKF/ZQkSiH7j/+I2OaP7rOOjX/NIMgyVVS2EDT0FnFUXOjAdkWIgNhXaxZ/MnHibzJpFX3YlA9Q0ALRf/E86PXAYpePtR37gHUlq0wYZmFgdo8E1aSRlgj/vTfAEU1RIGZhsXimxDGSqW91v0MoXsMJvlNxSFY8GTigmXY/VUXbEUESic9bcAikBuzrMPlqT9J0NmEzKg1O9T3uVCY6Q2TzPzwmlD5dubYZ6uV5FltDkdGscPOZ/LK7rs4GI3PckTqoZorflIIpbO0bfF49UTbKr9lmXk1xxEXIs1Fj6j9FJ6NF0f55ChGlTDeAq1HNfeIjEH4XPaKU/m+ZghWzde/JmK94f4mSInQt5pnar+IPFFk7PdSX6u/Kzy/iK+rAi+ViwTmJpUsApUB6JpXJjEX6wD85/OPMP9qDCr87uVSL6olX765W5Xr0HJ3JBoMwguLkLoOSyctj2u+qMF+J8Ghf1oZuyn4RFsmwX/j742qeLCt7M= 2022-09-05T02:25:20.083 INFO:teuthology.kill: smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOgGNyzfwqQOp8ZpM3+nim3V4de59155QSDh2jkZdX520zgxTd/RBQCSJwPsIrdBdANMQvHUlyHhVECJVmJ0d3U= 2022-09-05T02:25:20.083 INFO:teuthology.kill:2022-09-05 02:25:20,081.081 INFO:teuthology.nuke:Checking targets against current locks 2022-09-05T02:25:20.132 INFO:teuthology.kill:2022-09-05 02:25:20,132.132 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-05T02:25:20.165 INFO:teuthology.kill:2022-09-05 02:25:20,165.165 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-05T02:25:20.193 INFO:teuthology.kill:2022-09-05 02:25:20,193.193 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-05T02:25:20.227 INFO:teuthology.kill:2022-09-05 02:25:20,227.227 INFO:teuthology.orchestra.console:Power off smithi046 2022-09-05T02:25:20.273 INFO:teuthology.kill:2022-09-05 02:25:20,272.272 INFO:teuthology.orchestra.console:Power off smithi116 2022-09-05T02:25:20.308 INFO:teuthology.kill:2022-09-05 02:25:20,307.307 INFO:teuthology.orchestra.console:Power off smithi172 2022-09-05T02:25:28.603 INFO:teuthology.kill:2022-09-05 02:25:28,603.603 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2022-09-05T02:25:28.634 INFO:teuthology.kill:2022-09-05 02:25:28,634.634 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2022-09-05T02:25:28.667 INFO:teuthology.kill:2022-09-05 02:25:28,667.667 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2022-09-05T02:25:28.765 INFO:teuthology.kill:2022-09-05 02:25:28,764.764 INFO:teuthology.lock.ops:unlocked: smithi046.front.sepia.ceph.com 2022-09-05T02:25:28.790 INFO:teuthology.kill:2022-09-05 02:25:28,789.789 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com 2022-09-05T02:25:28.812 INFO:teuthology.kill:2022-09-05 02:25:28,811.811 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com 2022-09-05T02:27:29.111 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-05T02:27:29.244 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi116 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-04_23:15:18-rbd-wip-yuri10-testing-2022-09-04-0811-quincy-distro-default-smithi/7011379 2022-09-05T02:27:29.244 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi172 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-04_23:15:18-rbd-wip-yuri10-testing-2022-09-04-0811-quincy-distro-default-smithi/7011379