2022-09-30T04:46:44.873 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-09-30T04:46:45.089 INFO:teuthology.lock.ops:Start node 'smithi096.front.sepia.ceph.com' reimaging 2022-09-30T04:46:45.091 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: reset os type and version on server 2022-09-30T04:46:45.092 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2022-09-30T04:46:45.128 INFO:teuthology.lock.ops:Node 'smithi096.front.sepia.ceph.com' reimaging is complete 2022-09-30T04:46:45.128 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2022-09-30T04:46:45.129 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2022-09-30T04:46:45.129 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2022-09-30T04:46:45.154 INFO:teuthology.lock.ops:Node 'smithi112.front.sepia.ceph.com' reimaging is complete 2022-09-30T04:46:45.367 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 8.stream 2022-09-30T04:46:45.372 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of centos 8.stream 2022-09-30T04:46:45.777 INFO:teuthology.orchestra.console:Power off smithi096 2022-09-30T04:46:45.810 INFO:teuthology.orchestra.console:Power off smithi112 2022-09-30T04:46:45.956 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2022-09-30T04:46:46.058 INFO:teuthology.orchestra.console:Power on smithi096 2022-09-30T04:46:54.210 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2022-09-30T04:46:54.314 INFO:teuthology.orchestra.console:Power on smithi112 2022-09-30T04:46:54.549 INFO:teuthology.orchestra.console:Power on for smithi096 completed 2022-09-30T04:46:54.659 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish 2022-09-30T04:47:02.869 INFO:teuthology.orchestra.console:Power on for smithi112 completed 2022-09-30T04:47:02.971 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2022-09-30T04:50:58.255 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-30T04:51:23.289 INFO:teuthology.provision.fog.smithi096:Node is ready 2022-09-30T04:51:23.313 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com 2022-09-30T04:51:23.374 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096 2022-09-30T04:51:23.746 INFO:teuthology.provision.fog.smithi096:Deploy complete! 2022-09-30T04:51:40.916 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-30T04:51:41.172 INFO:teuthology.provision.fog.smithi112:Node is ready 2022-09-30T04:51:41.233 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2022-09-30T04:51:41.294 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2022-09-30T04:51:41.655 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2022-09-30T04:51:41.849 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com 2022-09-30T04:51:41.850 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-30T04:51:41.872 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2022-09-30T04:51:41.873 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-30T04:51:41.895 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server 2022-09-30T04:51:41.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2022-09-30T04:51:42.162 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2022-09-30T04:51:42.219 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2022-09-30T04:51:42.219 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8" 2022-09-30T04:51:42.220 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2022-09-30T04:51:42.220 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2022-09-30T04:51:42.220 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8" 2022-09-30T04:51:42.220 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8" 2022-09-30T04:51:42.221 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-30T04:51:42.221 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2022-09-30T04:51:42.221 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-30T04:51:42.221 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2022-09-30T04:51:42.222 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-30T04:51:42.222 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-30T04:51:42.222 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-30T04:51:42.223 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2022-09-30T04:51:42.245 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2022-09-30T04:51:42.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2022-09-30T04:51:42.487 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2022-09-30T04:51:42.545 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2022-09-30T04:51:42.545 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8" 2022-09-30T04:51:42.546 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2022-09-30T04:51:42.546 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2022-09-30T04:51:42.546 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8" 2022-09-30T04:51:42.547 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2022-09-30T04:51:42.547 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-30T04:51:42.547 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2022-09-30T04:51:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-30T04:51:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2022-09-30T04:51:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-30T04:51:42.548 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-30T04:51:42.549 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-30T04:51:42.549 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2022-09-30T04:51:42.665 INFO:teuthology.dispatcher.supervisor:Running job 7048632 2022-09-30T04:51:42.665 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-09-29_19:32:46-rados-wip-yuri6-testing-2022-09-29-0908-distro-default-smithi/7048632 --name yuriw-2022-09-29_19:32:46-rados-wip-yuri6-testing-2022-09-29-0908-distro-default-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} -- /home/teuthworker/archive/yuriw-2022-09-29_19:32:46-rados-wip-yuri6-testing-2022-09-29-0908-distro-default-smithi/7048632/orig.config.yaml 2022-09-30T04:51:42.671 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-09-29_19:32:46-rados-wip-yuri6-testing-2022-09-29-0908-distro-default-smithi/7048632 2022-09-30T04:51:42.672 INFO:teuthology.dispatcher.supervisor:Job PID: 13729 2022-09-30T04:51:42.672 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-30T16:52:51.615 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-30T16:52:51.799 INFO:teuthology.kill:Killing Pids: {13729} 2022-09-30T16:52:53.204 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-30T16:52:53.205 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-30T16:52:53.206 INFO:teuthology.misc:Compressing logs... 2022-09-30T16:52:53.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2022-09-30T16:52:53.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2022-09-30T16:52:53.745 INFO:teuthology.misc:Compressing logs... 2022-09-30T16:52:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-30T16:52:55.043 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-30T16:52:59.651 INFO:teuthology.misc:Compressing logs... 2022-09-30T16:57:09.025 INFO:teuthology.kill:No teuthology processes running 2022-09-30T16:57:10.390 INFO:teuthology.kill:Nuking machines: ['smithi096', 'smithi112'] 2022-09-30T16:57:11.564 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-30T16:57:11.566 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-30T16:57:11.566 INFO:teuthology.kill:2022-09-30 16:57:11,564.564 INFO:teuthology.nuke:targets: 2022-09-30T16:57:11.567 INFO:teuthology.kill: smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDohp5PaVvQSWet35ZPSbTHAnhy9L07cK731azGkUBox6Vwxk/f2Yiw/99tTYbeH6J06ig6mW5OEsJNRVKaii61+s71/trVst+fteeW02mXALnhWs4hm22EcZa4Y70C4GTDirVDwzrRYIHaA16QRRbH5A7VLxhL//MTyh2GA4p4UkbOZpcYCUS15rq1Md9qormfQmSZzjlHie7YxtxPrKpsTPsyUxNPNZi7/Emf4UZY7YmJGsdAUlAM8s6/6heqiweyYSTIzBfCw1OtydjBa93Hrw5edaKmdXhU3Mw9ReK2XctvpFsl/eVHrVH2+FsMp08WLCJGcVLYIsaJymn/qbQX2/t+P9bUF0KTalN0N4gTzNtXmBIgS5LyiGLKRCSyGkEtHhhvI1d5K7rny0Dsue2inONeIYNrODYwYIqovdKIJ22L9p+aBLXcwfIP2ApX1xIYyI988C4nE8eDvSt1FjTjyw2/XP8lZvEKFqMMqmr68QQ03OTtLuoo3K48Q453PoM= 2022-09-30T16:57:11.568 INFO:teuthology.kill: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPUed2M3aEdkTN92dKYRtW3GduursKaeEMQUk7HQz5r9Ks5rRaKKz3m9aQSzYKQGTKfQRZilknkZ8k64d89Wzys= 2022-09-30T16:57:11.568 INFO:teuthology.kill:2022-09-30 16:57:11,564.564 INFO:teuthology.nuke:Checking targets against current locks 2022-09-30T16:57:11.619 INFO:teuthology.kill:2022-09-30 16:57:11,619.619 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-30T16:57:11.647 INFO:teuthology.kill:2022-09-30 16:57:11,647.647 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-30T16:57:11.711 INFO:teuthology.kill:2022-09-30 16:57:11,710.710 INFO:teuthology.orchestra.console:Power off smithi096 2022-09-30T16:57:11.754 INFO:teuthology.kill:2022-09-30 16:57:11,754.754 INFO:teuthology.orchestra.console:Power off smithi112 2022-09-30T16:57:20.115 INFO:teuthology.kill:2022-09-30 16:57:20,115.115 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2022-09-30T16:57:20.264 INFO:teuthology.kill:2022-09-30 16:57:20,263.263 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2022-09-30T16:57:24.212 INFO:teuthology.kill:2022-09-30 16:57:24,211.211 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2022-09-30T16:57:24.369 INFO:teuthology.kill:2022-09-30 16:57:24,369.369 INFO:teuthology.lock.ops:unlocked: smithi096.front.sepia.ceph.com 2022-09-30T16:59:24.719 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-30T16:59:24.822 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi096 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-09-30_16:47:21-rgw-wip-cbodley-testing-distro-default-smithi/7049370 2022-09-30T16:59:24.823 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi112 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-09-30_16:47:21-rgw-wip-cbodley-testing-distro-default-smithi/7049369