2022-07-09T06:04:43.776 INFO:root:teuthology version: 1.1.1.dev381+g10062088 2022-07-09T06:04:44.023 INFO:teuthology.lock.ops:Start node 'smithi138.front.sepia.ceph.com' reimaging 2022-07-09T06:04:44.026 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: reset os type and version on server 2022-07-09T06:04:44.027 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-07-09T06:04:44.078 INFO:teuthology.lock.ops:Node 'smithi138.front.sepia.ceph.com' reimaging is complete 2022-07-09T06:04:44.078 INFO:teuthology.lock.ops:Start node 'smithi176.front.sepia.ceph.com' reimaging 2022-07-09T06:04:44.079 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: reset os type and version on server 2022-07-09T06:04:44.079 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2022-07-09T06:04:44.110 INFO:teuthology.lock.ops:Node 'smithi176.front.sepia.ceph.com' reimaging is complete 2022-07-09T06:04:44.342 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of rhel 8.6 2022-07-09T06:04:44.349 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of rhel 8.6 2022-07-09T06:04:44.822 INFO:teuthology.orchestra.console:Power off smithi176 2022-07-09T06:04:44.889 INFO:teuthology.orchestra.console:Power off smithi138 2022-07-09T06:04:53.314 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2022-07-09T06:04:53.330 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-07-09T06:04:53.431 INFO:teuthology.orchestra.console:Power on smithi176 2022-07-09T06:04:53.449 INFO:teuthology.orchestra.console:Power on smithi138 2022-07-09T06:04:57.738 INFO:teuthology.orchestra.console:Power on for smithi176 completed 2022-07-09T06:04:57.842 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish 2022-07-09T06:05:01.990 INFO:teuthology.orchestra.console:Power on for smithi138 completed 2022-07-09T06:05:02.092 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2022-07-09T06:07:33.508 ERROR:teuthology.orchestra.connection:Error authenticating with smithi138.front.sepia.ceph.com: Authentication failed. 2022-07-09T06:09:12.589 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-09T06:09:33.023 INFO:teuthology.provision.fog.smithi138:Node is ready 2022-07-09T06:09:33.060 INFO:teuthology.orchestra.run.smithi138.stdout:smithi086 2022-07-09T06:09:33.335 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2022-07-09T06:09:45.010 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-09T06:09:45.295 INFO:teuthology.provision.fog.smithi176:Node is ready 2022-07-09T06:09:45.340 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com 2022-07-09T06:09:45.448 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176 2022-07-09T06:09:45.927 INFO:teuthology.provision.fog.smithi176:Deploy complete! 2022-07-09T06:09:46.521 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2022-07-09T06:09:46.523 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-09T06:09:46.580 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com 2022-07-09T06:09:46.581 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-09T06:09:46.613 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2022-07-09T06:09:46.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-07-09T06:09:47.173 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2022-07-09T06:09:47.243 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="Red Hat Enterprise Linux" 2022-07-09T06:09:47.244 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8.6 (Ootpa)" 2022-07-09T06:09:47.244 INFO:teuthology.orchestra.run.smithi138.stdout:ID="rhel" 2022-07-09T06:09:47.245 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="fedora" 2022-07-09T06:09:47.245 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8.6" 2022-07-09T06:09:47.245 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2022-07-09T06:09:47.260 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-07-09T06:09:47.285 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2022-07-09T06:09:47.286 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-07-09T06:09:47.286 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.redhat.com/" 2022-07-09T06:09:47.287 INFO:teuthology.orchestra.run.smithi138.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-07-09T06:09:47.287 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-09T06:09:47.288 INFO:teuthology.orchestra.run.smithi138.stdout: 2022-07-09T06:09:47.288 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-09T06:09:47.289 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-07-09T06:09:47.289 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-07-09T06:09:47.296 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-07-09T06:09:47.298 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-07-09T06:09:47.385 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server 2022-07-09T06:09:47.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2022-07-09T06:09:47.983 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2022-07-09T06:09:48.062 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Red Hat Enterprise Linux" 2022-07-09T06:09:48.063 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="8.6 (Ootpa)" 2022-07-09T06:09:48.063 INFO:teuthology.orchestra.run.smithi176.stdout:ID="rhel" 2022-07-09T06:09:48.064 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="fedora" 2022-07-09T06:09:48.064 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="8.6" 2022-07-09T06:09:48.064 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el8" 2022-07-09T06:09:48.065 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-07-09T06:09:48.065 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2022-07-09T06:09:48.065 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-07-09T06:09:48.067 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.redhat.com/" 2022-07-09T06:09:48.068 INFO:teuthology.orchestra.run.smithi176.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-07-09T06:09:48.068 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-09T06:09:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: 2022-07-09T06:09:48.069 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-09T06:09:48.071 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-07-09T06:09:48.076 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-07-09T06:09:48.076 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-07-09T06:09:48.077 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2022-07-09T06:09:48.254 INFO:teuthology.dispatcher.supervisor:Running job 6921385 2022-07-09T06:09:48.254 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_10062088f503b43eff3624326bda825b23438f9b/virtualenv/bin/teuthology -v --owner scheduled_ksirivad@teuthology --archive /home/teuthworker/archive/ksirivad-2022-07-09_05:39:52-rados:thrash-erasure-code-main-distro-default-smithi/6921385 --name ksirivad-2022-07-09_05:39:52-rados:thrash-erasure-code-main-distro-default-smithi --description rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/ksirivad-2022-07-09_05:39:52-rados:thrash-erasure-code-main-distro-default-smithi/6921385/orig.config.yaml 2022-07-09T06:09:48.284 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ksirivad-2022-07-09_05:39:52-rados:thrash-erasure-code-main-distro-default-smithi/6921385 2022-07-09T06:09:48.287 INFO:teuthology.dispatcher.supervisor:Job PID: 2694 2022-07-09T06:09:48.288 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-09T18:10:17.580 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-09T18:10:17.729 INFO:teuthology.kill:Killing Pids: {2694} 2022-07-09T18:10:19.270 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-07-09T18:10:19.271 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-07-09T18:10:19.271 INFO:teuthology.misc:Compressing logs... 2022-07-09T18:10:19.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-07-09T18:10:19.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2022-07-09T18:10:19.794 INFO:teuthology.misc:Compressing logs... 2022-07-09T18:10:20.843 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-07-09T18:10:20.879 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-07-09T18:10:21.526 INFO:teuthology.misc:Compressing logs... 2022-07-09T18:10:28.993 INFO:teuthology.kill:No teuthology processes running 2022-07-09T18:10:30.684 INFO:teuthology.kill:Nuking machines: ['smithi138', 'smithi176'] 2022-07-09T18:10:31.927 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_10062088f503b43eff3624326bda825b23438f9b/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-09T18:10:31.928 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-09T18:10:31.929 INFO:teuthology.kill:2022-07-09 18:10:31,927.927 INFO:teuthology.nuke:targets: 2022-07-09T18:10:31.929 INFO:teuthology.kill: smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlEa4VeHJ5MJR0LlkKW+kzW8pg3QkwL85PLphmcSdgX5wfT79Hnsd3MW6OIcTUaSWEB7YvAjRl7Plf3/qemrsO8G0RofCPQ4qj64gr7AnI2t6A+qYOW56PDDcSOoPLt9qsnvwySdVF1kod/SN4gjCep9zO/lSfUrbctlLfnG/dQJlB8T2EIXJSz53SumW/t2q7AIASHmwxnfdJ44WhN0kqvn7mpFTlh3L7VY4u3fFYJ54WvMKVdt6SaMmZK3Um6COV0RWryypu38Rbwv/kSY0+GE5ktCNbzDNhQi91kwz55U46Q4i0Pi275ZfI81e2eCIG7ULgUk9FJf8xVzfaCqvK2aoSsneEm7eiXOqOr+NxAGJ+7M0wOocmwzniLIybjPvFhkExujO94WDDotTHxj2ksIi+wN6QSxJsyZjO5XXzPVW01jL0Y2nRPxOoRo2+kZfJfU0/6sPEHMPQiUMgo5xMlbDYdWtYHPSLsoxY0dB5xKhf4tQbp2Nu0ZSDQsmXfd0= 2022-07-09T18:10:31.929 INFO:teuthology.kill: smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcH3S6uPZ9M/kzSFbCrKLnkkQtDUUmCoLVdGv73u0mbAC1BAMHq469eBaisEh9hzWqSR6+jrGn25UNnCXPiQnYMQNWlXIvrXmt1PtiDeyLOhgOupqhQbJtOVTVZBGrfkwZu8MHinhNWLaFrkdUf4GbLz8Hq+HeNJLqFMDDPmQOMxt/MaY1zVIcW1eL8y1/cfJ5LnMQgAP/VHQQ+c76/9qhAbHekIAXTbxRTle7usWNKYMgaFiyZ7+HXFSGuHKe7sYJB61WkuNFRxd3Sms1Bo7r//KrQAa89YCKjs4WTu2iHnsiAnm12Yfs3MNSxza/BrnViKLMHemuKIZ6q0om2q1qzU/sfAzWvCxyHEqaMhdCASmlf8u1xv2Vr2hTiOwZEHVLvFPwJYgiNkTkD4VDQHl3YEUt8538137pQ9/ClxIcLZGy0H9MAA2Ya0V+d9HADjl8UAAlKRGdc8U6TVoU1cWCr7NlRh0vpZkbLokc2blx7isJjbILjhZoVueVGX9DHBc= 2022-07-09T18:10:31.930 INFO:teuthology.kill:2022-07-09 18:10:31,929.929 INFO:teuthology.nuke:Checking targets against current locks 2022-07-09T18:10:31.992 INFO:teuthology.kill:2022-07-09 18:10:31,992.992 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-09T18:10:32.032 INFO:teuthology.kill:2022-07-09 18:10:32,032.032 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-09T18:10:32.093 INFO:teuthology.kill:2022-07-09 18:10:32,092.092 INFO:teuthology.orchestra.console:Power off smithi138 2022-07-09T18:10:32.135 INFO:teuthology.kill:2022-07-09 18:10:32,135.135 INFO:teuthology.orchestra.console:Power off smithi176 2022-07-09T18:10:40.469 INFO:teuthology.kill:2022-07-09 18:10:40,468.468 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-07-09T18:10:40.629 INFO:teuthology.kill:2022-07-09 18:10:40,628.628 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com 2022-07-09T18:10:44.623 INFO:teuthology.kill:2022-07-09 18:10:44,622.622 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2022-07-09T18:10:44.810 INFO:teuthology.kill:2022-07-09 18:10:44,810.810 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com 2022-07-09T18:12:45.185 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15