2022-06-15T01:53:50.505 INFO:root:teuthology version: 1.1.1.dev370+g2290146e 2022-06-15T01:53:50.976 INFO:teuthology.lock.ops:Start node 'smithi059.front.sepia.ceph.com' reimaging 2022-06-15T01:53:50.978 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: reset os type and version on server 2022-06-15T01:53:50.978 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-06-15T01:53:51.007 INFO:teuthology.lock.ops:Node 'smithi059.front.sepia.ceph.com' reimaging is complete 2022-06-15T01:53:51.007 INFO:teuthology.lock.ops:Start node 'smithi120.front.sepia.ceph.com' reimaging 2022-06-15T01:53:51.008 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: reset os type and version on server 2022-06-15T01:53:51.009 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2022-06-15T01:53:51.056 INFO:teuthology.lock.ops:Node 'smithi120.front.sepia.ceph.com' reimaging is complete 2022-06-15T01:53:51.359 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of rhel 8.5 2022-06-15T01:53:51.399 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.5 2022-06-15T01:53:52.033 INFO:teuthology.orchestra.console:Power off smithi059 2022-06-15T01:53:52.145 INFO:teuthology.orchestra.console:Power off smithi120 2022-06-15T01:54:00.748 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-06-15T01:54:00.784 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2022-06-15T01:54:00.850 INFO:teuthology.orchestra.console:Power on smithi059 2022-06-15T01:54:00.894 INFO:teuthology.orchestra.console:Power on smithi120 2022-06-15T01:54:09.502 INFO:teuthology.orchestra.console:Power on for smithi120 completed 2022-06-15T01:54:09.552 INFO:teuthology.orchestra.console:Power on for smithi059 completed 2022-06-15T01:54:09.603 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish 2022-06-15T01:54:09.654 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2022-06-15T01:58:26.763 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-15T01:58:27.012 INFO:teuthology.provision.fog.smithi059:Node is ready 2022-06-15T01:58:27.032 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2022-06-15T01:58:27.103 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2022-06-15T01:58:27.477 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2022-06-15T01:58:41.844 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-15T01:58:42.065 INFO:teuthology.provision.fog.smithi120:Node is ready 2022-06-15T01:58:42.107 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com 2022-06-15T01:58:42.164 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120 2022-06-15T01:58:42.505 INFO:teuthology.provision.fog.smithi120:Deploy complete! 2022-06-15T01:58:42.727 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2022-06-15T01:58:42.728 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-15T01:58:42.755 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com 2022-06-15T01:58:42.756 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-15T01:58:42.777 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2022-06-15T01:58:42.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-06-15T01:58:43.141 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2022-06-15T01:58:43.207 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2022-06-15T01:58:43.208 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.5 (Ootpa)" 2022-06-15T01:58:43.208 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2022-06-15T01:58:43.209 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2022-06-15T01:58:43.209 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.5" 2022-06-15T01:58:43.210 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2022-06-15T01:58:43.210 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.5 (Ootpa)" 2022-06-15T01:58:43.211 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2022-06-15T01:58:43.211 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-06-15T01:58:43.212 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2022-06-15T01:58:43.212 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-06-15T01:58:43.212 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-15T01:58:43.213 INFO:teuthology.orchestra.run.smithi059.stdout: 2022-06-15T01:58:43.217 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-15T01:58:43.218 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.5 2022-06-15T01:58:43.218 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-06-15T01:58:43.218 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.5" 2022-06-15T01:58:43.220 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-06-15T01:58:43.255 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server 2022-06-15T01:58:43.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2022-06-15T01:58:43.504 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2022-06-15T01:58:43.561 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux" 2022-06-15T01:58:43.561 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.5 (Ootpa)" 2022-06-15T01:58:43.561 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel" 2022-06-15T01:58:43.562 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora" 2022-06-15T01:58:43.562 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.5" 2022-06-15T01:58:43.562 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8" 2022-06-15T01:58:43.562 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.5 (Ootpa)" 2022-06-15T01:58:43.563 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2022-06-15T01:58:43.563 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-06-15T01:58:43.563 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/" 2022-06-15T01:58:43.563 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-06-15T01:58:43.563 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-15T01:58:43.564 INFO:teuthology.orchestra.run.smithi120.stdout: 2022-06-15T01:58:43.564 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-15T01:58:43.564 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.5 2022-06-15T01:58:43.564 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-06-15T01:58:43.565 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.5" 2022-06-15T01:58:43.565 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2022-06-15T01:58:43.715 INFO:teuthology.dispatcher.supervisor:Running job 6878951 2022-06-15T01:58:43.716 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_2290146eac7577b8500f128a53856c3ea4a00e3c/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-06-14_21:37:13-rados:thrash-erasure-code-overwrites-main-distro-default-smithi/6878951 --name lflores-2022-06-14_21:37:13-rados:thrash-erasure-code-overwrites-main-distro-default-smithi --description rados:thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} -- /home/teuthworker/archive/lflores-2022-06-14_21:37:13-rados:thrash-erasure-code-overwrites-main-distro-default-smithi/6878951/orig.config.yaml 2022-06-15T01:58:43.742 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-06-14_21:37:13-rados:thrash-erasure-code-overwrites-main-distro-default-smithi/6878951 2022-06-15T01:58:43.743 INFO:teuthology.dispatcher.supervisor:Job PID: 14779 2022-06-15T01:58:43.744 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-06-15T13:59:50.188 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-06-15T13:59:50.601 INFO:teuthology.kill:Killing Pids: {14779} 2022-06-15T13:59:58.053 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-06-15T13:59:58.087 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-06-15T13:59:58.088 INFO:teuthology.misc:Compressing logs... 2022-06-15T13:59:58.124 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-06-15T13:59:58.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2022-06-15T13:59:59.693 INFO:teuthology.misc:Compressing logs... 2022-06-15T14:00:00.698 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-15T14:00:01.061 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-15T14:00:05.260 INFO:teuthology.misc:Compressing logs... 2022-06-15T14:00:37.200 INFO:teuthology.kill:No teuthology processes running 2022-06-15T14:00:44.395 INFO:teuthology.kill:Nuking machines: ['smithi059', 'smithi120'] 2022-06-15T14:00:49.433 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_2290146eac7577b8500f128a53856c3ea4a00e3c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-06-15T14:00:49.447 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-06-15T14:00:49.447 INFO:teuthology.kill:2022-06-15 14:00:49,433.433 INFO:teuthology.nuke:targets: 2022-06-15T14:00:49.448 INFO:teuthology.kill: smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC89+FyYZ7G7hY6bgnsjM7hYXnJPMfp899Al8QuO0DVlsoS2ONliKgB6ED0W5eApV0zQgFddG+0ts/nUC0iEtM47/SMdt3VQmlHfDMFgA19uJ349ic/h0DD8a9/eJsp26iaMxLSZTNAi+vobYFEieFlao4Ve3zaHfUQ2A9JMsfU55r3WGYg2FQNgn5oeuA2jWiZJ2SsbjswQq/H5PStKCp9Hrb6HfR/ICyZiP1O3mzaYUByxvBknxHSqNr1PmVPEHCIH473B1eHlliFpDcbtoCwckJyq1ShE+hQCnrnNHXenmzHu3ejUPzNUwE3e4EZ8KSVN5sZWi3h8v6XZG1zS5GyIN6oQLjaDBBfL48345JvzBBKZhiwtA5YRqMZGzt5eIE3PnXt7bK1HoZQ941UjqtRthJIfGy1A9w4doyKf0z/V/5+RaUP1BEPcMgaKcOubCOKxzNuO1dTshEZjs4vcwzpBphZMr2j1yBUkx485l9Q3RWCaz++WU0HLlrdbEgdryU= 2022-06-15T14:00:49.448 INFO:teuthology.kill: smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPo6sjARVdTE8KZXVpB5J89udLmF6B6VvCgphGEJZkX2Lzhzp579cIXsagXbnFdKC1Iaus4zcDgSkOz8oOZqqYfeBEFdZ/3GBRLcbJ4V0NO+nzVvUmcGDLGCXgcIeM4qNrUr12yBQnqO650htx3NXYglvsayfeFjSLgjoRG4HG7nolTKZF7ZrWMtRv60ul9gBL9cTSANcKLpZ6qywBwthgQchQo9mu8/HqRkZt004WXIONcbKdMUdGYEck5YCvaCwS+da3RqmNbYNDvKK7JYCTLJydDa6e/x/rZ0vTPmwZNIbCtdxfHMlA6IVeiIrX10P8YYj3XeYZ6krl+j9xBoEo7YB0nTd22j/Bvn7eeKq+X8exxxJDsSLHlEWfUSmyoaUSBfSgSKIO5njWOr5fEbF8FpePo87sR8fq/DmxaxovrrAemJkmkdtgRZq2Ua6ntP3FAgPCeyYdD88FPgmFNKGgWy0opByScQ7z+gle9vMVUXToSTf2fTXwTix1ajfu79E= 2022-06-15T14:00:49.484 INFO:teuthology.kill:2022-06-15 14:00:49,483.483 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-15T14:00:49.508 INFO:teuthology.kill:2022-06-15 14:00:49,507.507 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-15T14:00:49.790 INFO:teuthology.kill:2022-06-15 14:00:49,777.777 INFO:teuthology.orchestra.console:Power off smithi120 2022-06-15T14:00:49.930 INFO:teuthology.kill:2022-06-15 14:00:49,928.928 INFO:teuthology.orchestra.console:Power off smithi059 2022-06-15T14:00:58.473 INFO:teuthology.kill:2022-06-15 14:00:58,473.473 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2022-06-15T14:00:58.678 INFO:teuthology.kill:2022-06-15 14:00:58,661.661 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com 2022-06-15T14:01:02.750 INFO:teuthology.kill:2022-06-15 14:01:02,737.737 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-06-15T14:01:02.970 INFO:teuthology.kill:2022-06-15 14:01:02,961.961 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2022-06-15T14:03:03.711 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-06-15T14:03:03.909 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi059 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-06-15_03:30:04-rados-octopus-distro-default-smithi/6879389 2022-06-15T14:03:03.910 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi120 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-06-15_03:30:04-rados-octopus-distro-default-smithi/6879385