2022-06-28T18:56:23.872 INFO:root:teuthology version: 1.1.1.dev376+g082ae7ef 2022-06-28T18:56:24.061 INFO:teuthology.lock.ops:Start node 'smithi070.front.sepia.ceph.com' reimaging 2022-06-28T18:56:24.063 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: reset os type and version on server 2022-06-28T18:56:24.063 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2022-06-28T18:56:24.089 INFO:teuthology.lock.ops:Node 'smithi070.front.sepia.ceph.com' reimaging is complete 2022-06-28T18:56:24.089 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging 2022-06-28T18:56:24.090 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server 2022-06-28T18:56:24.090 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-06-28T18:56:24.115 INFO:teuthology.lock.ops:Node 'smithi121.front.sepia.ceph.com' reimaging is complete 2022-06-28T18:56:24.341 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of centos 8.stream 2022-06-28T18:56:24.345 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of centos 8.stream 2022-06-28T18:56:24.778 INFO:teuthology.orchestra.console:Power off smithi121 2022-06-28T18:56:24.791 INFO:teuthology.orchestra.console:Power off smithi070 2022-06-28T18:56:33.156 INFO:teuthology.orchestra.console:Power off for smithi070 completed 2022-06-28T18:56:33.258 INFO:teuthology.orchestra.console:Power on smithi070 2022-06-28T18:56:37.273 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-06-28T18:56:37.374 INFO:teuthology.orchestra.console:Power on smithi121 2022-06-28T18:56:41.626 INFO:teuthology.orchestra.console:Power on for smithi070 completed 2022-06-28T18:56:41.629 INFO:teuthology.orchestra.console:Power on for smithi121 completed 2022-06-28T18:56:41.728 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish 2022-06-28T18:56:41.733 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2022-06-28T19:00:51.615 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-28T19:01:11.960 INFO:teuthology.provision.fog.smithi070:Node is ready 2022-06-28T19:01:11.981 INFO:teuthology.orchestra.run.smithi070.stdout:smithi149 2022-06-28T19:01:12.040 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149 2022-06-28T19:01:12.432 INFO:teuthology.provision.fog.smithi070:Deploy complete! 2022-06-28T19:01:29.262 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-28T19:01:29.590 INFO:teuthology.provision.fog.smithi121:Node is ready 2022-06-28T19:01:29.609 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com 2022-06-28T19:01:29.670 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121 2022-06-28T19:01:30.069 INFO:teuthology.provision.fog.smithi121:Deploy complete! 2022-06-28T19:01:30.294 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com 2022-06-28T19:01:30.295 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-28T19:01:30.321 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com 2022-06-28T19:01:30.322 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-28T19:01:30.344 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server 2022-06-28T19:01:30.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2022-06-28T19:01:30.564 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2022-06-28T19:01:30.626 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2022-06-28T19:01:30.626 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8" 2022-06-28T19:01:30.627 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2022-06-28T19:01:30.627 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2022-06-28T19:01:30.627 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8" 2022-06-28T19:01:30.627 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8" 2022-06-28T19:01:30.628 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-28T19:01:30.628 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2022-06-28T19:01:30.628 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-28T19:01:30.628 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2022-06-28T19:01:30.629 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-28T19:01:30.629 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-28T19:01:30.629 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-28T19:01:30.631 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2022-06-28T19:01:30.657 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server 2022-06-28T19:01:30.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-06-28T19:01:30.884 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2022-06-28T19:01:30.945 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2022-06-28T19:01:30.945 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8" 2022-06-28T19:01:30.946 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2022-06-28T19:01:30.946 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2022-06-28T19:01:30.947 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8" 2022-06-28T19:01:30.947 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2022-06-28T19:01:30.948 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-28T19:01:30.948 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2022-06-28T19:01:30.949 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-28T19:01:30.949 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2022-06-28T19:01:30.949 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-28T19:01:30.950 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-28T19:01:30.950 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-28T19:01:30.951 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-06-28T19:01:31.092 INFO:teuthology.dispatcher.supervisor:Running job 6901699 2022-06-28T19:01:31.092 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_082ae7ef4302fa54665ed0a2535e8e254118dcfd/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi/6901699 --name lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi --description rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi/6901699/orig.config.yaml 2022-06-28T19:01:31.097 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi/6901699 2022-06-28T19:01:31.098 INFO:teuthology.dispatcher.supervisor:Job PID: 27595 2022-06-28T19:01:31.098 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-06-29T07:03:12.720 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-06-29T07:03:13.433 INFO:teuthology.kill:Killing Pids: {27595} 2022-06-29T07:03:15.420 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-06-29T07:03:15.424 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-06-29T07:03:15.432 INFO:teuthology.misc:Compressing logs... 2022-06-29T07:03:15.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2022-06-29T07:03:16.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-06-29T07:03:17.795 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-29T07:03:17.939 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-29T07:03:18.946 INFO:teuthology.kill:No teuthology processes running 2022-06-29T07:03:20.931 INFO:teuthology.kill:Nuking machines: ['smithi070', 'smithi121'] 2022-06-29T07:03:22.579 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_082ae7ef4302fa54665ed0a2535e8e254118dcfd/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-06-29T07:03:22.581 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-06-29T07:03:22.581 INFO:teuthology.kill:2022-06-29 07:03:22,579.579 INFO:teuthology.nuke:targets: 2022-06-29T07:03:22.582 INFO:teuthology.kill: smithi070.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfaWBPasvJ7DkHQYx06z8RlhZotI/p+qDX8oRNL67aMQW/sCVLoIpds02jbnD4usPX3KXaJhIkG0IPNlxdXGRQtBhKy49/IKxU4HPPv+OQMUYKUIHcO+F+VCTHvRh2H8JWOPsawxuzMstPqrMLus64eQykqkrRsZLSsoLzerM1kNDuGSOVmS4YfRjgTzWWe+jInPGl9o7Wh9zj5JFTIJeZ0AD2kPzI+YzLqtxLK8Lop4MBfUanxRM4d8Bdh5eplRt4J6R/4KXUj/PTc7745Dl4L35WCMbtlozQWW1oojpIji3MQv9L3KFQgpw8nz66e/m1kwbynO+gfkvd96qWgNb6t3YjcjYJ9qaBI1lWrlFte/g1sValVIyGFSNjNFJPVcLqqQm+t6A/3LX+M9h/XtY0391UR7FIswibi4b0st5Md/RtO5qY/C6bOdF1+cAv+6jVrElY1IEjqONRoHHF/XTXligjAy0kPsGs/7aFzeuksBgAC5N904yEqv+cewlHodE= 2022-06-29T07:03:22.582 INFO:teuthology.kill: smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkk7RZ9Xmiax2cZNWowXNVDT9ykuVBbpqITg3UqyLfS8pEVY1i1kz6gR9QMt4qOS5v3X/yquD3ZZmvQENBj+jgY8nnaQKm15qnomq2uHXmTGOoP9DQyf9l8TyTTHySH6lx6/h9puwefZsU7+5HDgNs95pCNMyDQRSPj8nQfRCKCoal3wN9XiU177d2QposnBh84p+qgIBueSs2KqETyr3jahb6fXro2a10rsq3QNZiYSOgPuWgqqXwq9pZmp4R5gPECxVImgF7P0Xr9MUKFIIrg5E6NWDuKITtila8TGQryfa2OoZlwCu82XQfVCwDiHNYQ/uGcsmAxqgMjHYFTvhhi1C2Wgkk+AENxSQMZVEf1hSOZoZ285WRUyekl7qLcO+IlBLoYRibymN3AUg6oOo87g0pfmqjT4dOfg5oFkoWnmG562TD1GKlFVJ07oB4nlNIMk1P7emcKNzgCrm20Wn2kMDyoYvNT3B5/fXtL8ZKlZD1JelXz80aABj/lf/8HCE= 2022-06-29T07:03:22.583 INFO:teuthology.kill:2022-06-29 07:03:22,583.583 INFO:teuthology.nuke:Checking targets against current locks 2022-06-29T07:03:22.653 INFO:teuthology.kill:2022-06-29 07:03:22,653.653 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-29T07:03:22.682 INFO:teuthology.kill:2022-06-29 07:03:22,682.682 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-29T07:03:22.805 INFO:teuthology.kill:2022-06-29 07:03:22,804.804 INFO:teuthology.orchestra.console:Power off smithi070 2022-06-29T07:03:22.915 INFO:teuthology.kill:2022-06-29 07:03:22,915.915 INFO:teuthology.orchestra.console:Power off smithi121 2022-06-29T07:03:31.306 INFO:teuthology.kill:2022-06-29 07:03:31,305.305 INFO:teuthology.orchestra.console:Power off for smithi070 completed 2022-06-29T07:03:31.379 INFO:teuthology.kill:2022-06-29 07:03:31,379.379 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-06-29T07:03:31.466 INFO:teuthology.kill:2022-06-29 07:03:31,465.465 INFO:teuthology.lock.ops:unlocked: smithi070.front.sepia.ceph.com 2022-06-29T07:03:31.537 INFO:teuthology.kill:2022-06-29 07:03:31,536.536 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com 2022-06-29T07:05:32.284 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-06-29T07:05:32.495 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi070 but it was locked by another job: /home/teuthworker/archive/nshelke-2022-06-29_07:00:00-rados:singleton-bluestore-wip-yuri6-testing-2022-06-22-1419-distro-default-smithi/6904647 2022-06-29T07:05:32.497 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi121 but it was locked by another job: /home/teuthworker/archive/nshelke-2022-06-29_07:00:00-rados:singleton-bluestore-wip-yuri6-testing-2022-06-22-1419-distro-default-smithi/6904646