2022-06-26T04:52:54.316 INFO:root:teuthology version: 1.1.1.dev376+g082ae7ef
2022-06-26T04:52:54.464 INFO:teuthology.lock.ops:Start node 'smithi067.front.sepia.ceph.com' reimaging
2022-06-26T04:52:54.465 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: reset os type and version on server
2022-06-26T04:52:54.466 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2022-06-26T04:52:54.485 INFO:teuthology.lock.ops:Node 'smithi067.front.sepia.ceph.com' reimaging is complete
2022-06-26T04:52:54.486 INFO:teuthology.lock.ops:Start node 'smithi142.front.sepia.ceph.com' reimaging
2022-06-26T04:52:54.486 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: reset os type and version on server
2022-06-26T04:52:54.487 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server
2022-06-26T04:52:54.510 INFO:teuthology.lock.ops:Node 'smithi142.front.sepia.ceph.com' reimaging is complete
2022-06-26T04:52:54.726 INFO:teuthology.provision.fog.smithi067:Scheduling deploy of rhel 8.4
2022-06-26T04:52:54.738 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of rhel 8.4
2022-06-26T04:52:55.193 INFO:teuthology.orchestra.console:Power off smithi142
2022-06-26T04:52:55.317 INFO:teuthology.orchestra.console:Power off smithi067
2022-06-26T04:53:03.566 INFO:teuthology.orchestra.console:Power off for smithi142 completed
2022-06-26T04:53:03.668 INFO:teuthology.orchestra.console:Power on smithi142
2022-06-26T04:53:07.798 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2022-06-26T04:53:07.903 INFO:teuthology.orchestra.console:Power on smithi067
2022-06-26T04:53:12.082 INFO:teuthology.orchestra.console:Power on for smithi142 completed
2022-06-26T04:53:12.183 INFO:teuthology.orchestra.console:Power on for smithi067 completed
2022-06-26T04:53:12.184 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish
2022-06-26T04:53:12.284 INFO:teuthology.provision.fog.smithi067:Waiting for deploy to finish
2022-06-26T04:55:44.109 ERROR:teuthology.orchestra.connection:Error authenticating with smithi142.front.sepia.ceph.com: Authentication failed.
2022-06-26T04:57:23.030 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-06-26T04:57:43.143 INFO:teuthology.provision.fog.smithi142:Node is ready
2022-06-26T04:57:43.177 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142.front.sepia.ceph.com
2022-06-26T04:57:43.247 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142 smithi142.front.sepia.ceph.com smithi142
2022-06-26T04:57:43.617 INFO:teuthology.provision.fog.smithi142:Deploy complete!
2022-06-26T04:58:08.740 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-06-26T04:58:08.990 INFO:teuthology.provision.fog.smithi067:Node is ready
2022-06-26T04:58:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067.front.sepia.ceph.com
2022-06-26T04:58:09.065 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67 smithi067.front.sepia.ceph.com smithi067
2022-06-26T04:58:09.401 INFO:teuthology.provision.fog.smithi067:Deploy complete!
2022-06-26T04:58:09.630 INFO:teuthology.lock.ops:Checking smithi067.front.sepia.ceph.com
2022-06-26T04:58:09.632 INFO:teuthology.lock.ops:New key found. Updating...
2022-06-26T04:58:09.660 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com
2022-06-26T04:58:09.660 INFO:teuthology.lock.ops:New key found. Updating...
2022-06-26T04:58:09.682 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: set os type and version on server
2022-06-26T04:58:09.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com'
2022-06-26T04:58:10.034 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64
2022-06-26T04:58:10.097 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="Red Hat Enterprise Linux"
2022-06-26T04:58:10.098 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8.4 (Ootpa)"
2022-06-26T04:58:10.098 INFO:teuthology.orchestra.run.smithi067.stdout:ID="rhel"
2022-06-26T04:58:10.099 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="fedora"
2022-06-26T04:58:10.099 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8.4"
2022-06-26T04:58:10.100 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8"
2022-06-26T04:58:10.100 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-06-26T04:58:10.101 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31"
2022-06-26T04:58:10.106 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-06-26T04:58:10.108 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://www.redhat.com/"
2022-06-26T04:58:10.109 INFO:teuthology.orchestra.run.smithi067.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-06-26T04:58:10.109 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-06-26T04:58:10.109 INFO:teuthology.orchestra.run.smithi067.stdout:
2022-06-26T04:58:10.110 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-06-26T04:58:10.110 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-06-26T04:58:10.130 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-06-26T04:58:10.131 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-06-26T04:58:10.133 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2022-06-26T04:58:10.187 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server
2022-06-26T04:58:10.187 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com'
2022-06-26T04:58:10.542 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64
2022-06-26T04:58:10.620 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Red Hat Enterprise Linux"
2022-06-26T04:58:10.621 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8.4 (Ootpa)"
2022-06-26T04:58:10.621 INFO:teuthology.orchestra.run.smithi142.stdout:ID="rhel"
2022-06-26T04:58:10.621 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="fedora"
2022-06-26T04:58:10.621 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8.4"
2022-06-26T04:58:10.622 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8"
2022-06-26T04:58:10.622 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-06-26T04:58:10.622 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31"
2022-06-26T04:58:10.622 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-06-26T04:58:10.623 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.redhat.com/"
2022-06-26T04:58:10.623 INFO:teuthology.orchestra.run.smithi142.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-06-26T04:58:10.623 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-06-26T04:58:10.623 INFO:teuthology.orchestra.run.smithi142.stdout:
2022-06-26T04:58:10.624 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-06-26T04:58:10.624 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-06-26T04:58:10.624 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-06-26T04:58:10.624 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-06-26T04:58:10.626 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server
2022-06-26T04:58:10.755 INFO:teuthology.dispatcher.supervisor:Running job 6898719
2022-06-26T04:58:10.759 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_082ae7ef4302fa54665ed0a2535e8e254118dcfd/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-06-26_03:31:02-rados-pacific-distro-default-smithi/6898719 --name teuthology-2022-06-26_03:31:02-rados-pacific-distro-default-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects} -- /home/teuthworker/archive/teuthology-2022-06-26_03:31:02-rados-pacific-distro-default-smithi/6898719/orig.config.yaml
2022-06-26T04:58:10.773 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-06-26_03:31:02-rados-pacific-distro-default-smithi/6898719
2022-06-26T04:58:10.775 INFO:teuthology.dispatcher.supervisor:Job PID: 30046
2022-06-26T04:58:10.775 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-06-26T16:58:37.182 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-06-26T16:58:37.741 INFO:teuthology.kill:Killing Pids: {30046}
2022-06-26T16:58:39.237 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2022-06-26T16:58:39.238 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2022-06-26T16:58:39.238 INFO:teuthology.misc:Compressing logs...
2022-06-26T16:58:39.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com'
2022-06-26T16:58:39.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com'
2022-06-26T16:58:39.781 INFO:teuthology.misc:Compressing logs...
2022-06-26T16:58:40.763 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-06-26T16:58:41.127 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-06-26T16:58:41.776 INFO:teuthology.misc:Compressing logs...
2022-06-26T16:58:44.696 INFO:teuthology.kill:No teuthology processes running
2022-06-26T16:58:46.092 INFO:teuthology.kill:Nuking machines: ['smithi067', 'smithi142']
2022-06-26T16:58:47.161 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_082ae7ef4302fa54665ed0a2535e8e254118dcfd/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-06-26T16:58:47.162 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-06-26T16:58:47.162 INFO:teuthology.kill:2022-06-26 16:58:47,160.160 INFO:teuthology.nuke:targets:
2022-06-26T16:58:47.162 INFO:teuthology.kill:  smithi067.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRo4quW0mOnJLZ2cIqmw3D2NKXzzMa4rcdpBviLi9hjWnNCpvqJfJu2TK4IB1xEmQ80tgvcMltnRAtGYwdC/oQV8ffjfxrlD8MDVYtcG0kPP+GnEIn4JptG8jvofqZQPDjEbtbKK5RKi3aWZ7feRd5pdLE4ZKEWaNczv1GiHJdnT6MfK1mevxJLfiqV28FS0+6aCMtK3cstR28+WmRILO5S8GQd0PNxZSkNuKf099AJ0CTCrh3kj/rAunMgjOLxSvpqlyPRd6BRgXpma/pcHbzp+PYe9OIdOTiqhzPP4DwjjMAVrrZvtr73L/GGjdUOlrqUohJReGim02ynBn4y6Gm9rNQLJkQH5VqDAvUM0IIVjVsn0fO/3/+lt+alWn+hYP4QEQxAz+eui1Uw2rnK9VYpE2KW0Jp0StlwvTqFIk4ajNQ+/BaTN9YMVdv4S4z+P03CzntDopioVOFRJXpArRZFZn6ekyPUDixnDUdhwOPuZ3DB+gEulJvjyvYIqO4RC0=
2022-06-26T16:58:47.163 INFO:teuthology.kill:  smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD2jvHRSKfVhS8FsI3JKrMiRlfLOuTyGg+X6lcTKOx6wpTTrZR2iDrO47qPDFHQzIHXNoo9ahcuT1CeQzbQSwODwE7/uJghWPk3VrmzS6rLjQT5BcypP7xG1+1OSd6hKJ/3OiLnn2QeDFJzUdT+xWKj0WWSXCsuNGERw2DGDS7sTmPTzyFY9e325/+hBDB0G+lJ29iLNhGVd1g8KktpJfLnjC81miOFTZ0coBpxvb98VXF979hji4rGtoJ0+834Y4qBzI5bSFAOcGkgja7KDtoEycYNUPVaOA3Uim08PaXUrkOJBSZmTC+FfG5Qz2ykEuFVWk4qRprB4oO88n8qU1Q3mm8MLUJBnkABSALDMV1joz0qCOA2GMLrXrLSjuEJw+i9H8hF2aZxulz+QLfO8rt7E2mpMvhO8LcffOjz9q+G73ePl21syxx8zIvbLXET2teJAJ2RAoa2eagR6ot7ggDP2RWl3bEgD36T39oYu4rxXsyrzkTvF18zx7PwDpfntas=
2022-06-26T16:58:47.163 INFO:teuthology.kill:2022-06-26 16:58:47,161.161 INFO:teuthology.nuke:Checking targets against current locks
2022-06-26T16:58:47.214 INFO:teuthology.kill:2022-06-26 16:58:47,214.214 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-06-26T16:58:47.241 INFO:teuthology.kill:2022-06-26 16:58:47,241.241 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-06-26T16:58:47.294 INFO:teuthology.kill:2022-06-26 16:58:47,293.293 INFO:teuthology.orchestra.console:Power off smithi067
2022-06-26T16:58:47.334 INFO:teuthology.kill:2022-06-26 16:58:47,333.333 INFO:teuthology.orchestra.console:Power off smithi142
2022-06-26T16:58:55.693 INFO:teuthology.kill:2022-06-26 16:58:55,692.692 INFO:teuthology.orchestra.console:Power off for smithi142 completed
2022-06-26T16:58:55.844 INFO:teuthology.kill:2022-06-26 16:58:55,843.843 INFO:teuthology.lock.ops:unlocked: smithi142.front.sepia.ceph.com
2022-06-26T16:58:59.782 INFO:teuthology.kill:2022-06-26 16:58:59,782.782 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2022-06-26T16:58:59.932 INFO:teuthology.kill:2022-06-26 16:58:59,931.931 INFO:teuthology.lock.ops:unlocked: smithi067.front.sepia.ceph.com
2022-06-26T17:01:00.136 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15