2022-02-07T15:13:16.843 INFO:root:teuthology version: 1.1.0-3094160c
2022-02-07T15:13:16.961 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging
2022-02-07T15:13:16.962 INFO:teuthology.lock.ops:Updating [ubuntu@smithi029.front.sepia.ceph.com]: reset os type and version on server
2022-02-07T15:13:16.963 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server
2022-02-07T15:13:16.981 INFO:teuthology.lock.ops:Node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging is complete
2022-02-07T15:13:16.981 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging
2022-02-07T15:13:16.981 INFO:teuthology.lock.ops:Updating [ubuntu@smithi121.front.sepia.ceph.com]: reset os type and version on server
2022-02-07T15:13:16.982 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2022-02-07T15:13:17.001 INFO:teuthology.lock.ops:Node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging is complete
2022-02-07T15:13:17.216 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of rhel 8.4
2022-02-07T15:13:17.244 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of rhel 8.4
2022-02-07T15:13:17.656 INFO:teuthology.orchestra.console:Power off smithi121
2022-02-07T15:13:17.700 INFO:teuthology.orchestra.console:Power off smithi029
2022-02-07T15:13:26.064 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2022-02-07T15:13:26.165 INFO:teuthology.orchestra.console:Power on smithi029
2022-02-07T15:13:30.143 INFO:teuthology.orchestra.console:Power off for smithi121 completed
2022-02-07T15:13:30.245 INFO:teuthology.orchestra.console:Power on smithi121
2022-02-07T15:13:34.495 INFO:teuthology.orchestra.console:Power on for smithi121 completed
2022-02-07T15:13:34.525 INFO:teuthology.orchestra.console:Power on for smithi029 completed
2022-02-07T15:13:34.597 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish
2022-02-07T15:13:34.626 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish
2022-02-07T15:16:21.243 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: Authentication failed.
2022-02-07T15:18:01.176 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-07T15:18:11.481 INFO:teuthology.provision.fog.smithi029:Node is ready
2022-02-07T15:18:11.541 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com
2022-02-07T15:18:11.599 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029
2022-02-07T15:18:11.969 INFO:teuthology.provision.fog.smithi029:Deploy complete!
2022-02-07T15:18:21.770 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-07T15:18:21.987 INFO:teuthology.provision.fog.smithi121:Node is ready
2022-02-07T15:18:22.002 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com
2022-02-07T15:18:22.058 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121
2022-02-07T15:18:22.378 INFO:teuthology.provision.fog.smithi121:Deploy complete!
2022-02-07T15:18:22.532 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com
2022-02-07T15:18:22.533 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-07T15:18:22.556 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com
2022-02-07T15:18:22.556 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-07T15:18:22.575 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server
2022-02-07T15:18:22.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com'
2022-02-07T15:18:22.815 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64
2022-02-07T15:18:22.872 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Red Hat Enterprise Linux"
2022-02-07T15:18:22.872 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8.4 (Ootpa)"
2022-02-07T15:18:22.873 INFO:teuthology.orchestra.run.smithi029.stdout:ID="rhel"
2022-02-07T15:18:22.873 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="fedora"
2022-02-07T15:18:22.873 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8.4"
2022-02-07T15:18:22.873 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8"
2022-02-07T15:18:22.874 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-02-07T15:18:22.874 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31"
2022-02-07T15:18:22.874 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-02-07T15:18:22.874 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.redhat.com/"
2022-02-07T15:18:22.874 INFO:teuthology.orchestra.run.smithi029.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-02-07T15:18:22.875 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-07T15:18:22.875 INFO:teuthology.orchestra.run.smithi029.stdout:
2022-02-07T15:18:22.875 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-07T15:18:22.875 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-02-07T15:18:22.876 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-02-07T15:18:22.876 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-02-07T15:18:22.877 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server
2022-02-07T15:18:22.896 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server
2022-02-07T15:18:22.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com'
2022-02-07T15:18:23.119 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64
2022-02-07T15:18:23.172 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux"
2022-02-07T15:18:23.173 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.4 (Ootpa)"
2022-02-07T15:18:23.173 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel"
2022-02-07T15:18:23.173 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora"
2022-02-07T15:18:23.174 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.4"
2022-02-07T15:18:23.174 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8"
2022-02-07T15:18:23.174 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-02-07T15:18:23.174 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31"
2022-02-07T15:18:23.175 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-02-07T15:18:23.175 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/"
2022-02-07T15:18:23.175 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-02-07T15:18:23.175 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-07T15:18:23.175 INFO:teuthology.orchestra.run.smithi121.stdout:
2022-02-07T15:18:23.176 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-07T15:18:23.176 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-02-07T15:18:23.176 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-02-07T15:18:23.176 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-02-07T15:18:23.177 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2022-02-07T15:18:23.271 INFO:teuthology.dispatcher.supervisor:Running job 6666515
2022-02-07T15:18:23.272 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666515 --name benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} -- /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666515/orig.config.yaml
2022-02-07T15:18:23.276 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666515
2022-02-07T15:18:23.276 INFO:teuthology.dispatcher.supervisor:Job PID: 23814
2022-02-07T15:18:23.277 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-07T21:48:34.154 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-07T21:48:34.355 INFO:teuthology.kill:Killing Pids: {23814}
2022-02-07T21:48:35.629 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2022-02-07T21:48:35.630 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2022-02-07T21:48:35.631 INFO:teuthology.misc:Compressing logs...
2022-02-07T21:48:35.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com'
2022-02-07T21:48:35.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com'
2022-02-07T21:48:36.183 INFO:teuthology.misc:Compressing logs...
2022-02-07T21:48:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-07T21:48:36.904 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-07T21:48:37.288 INFO:teuthology.misc:Compressing logs...
2022-02-07T21:48:41.227 INFO:teuthology.kill:No teuthology processes running
2022-02-07T21:48:42.457 INFO:teuthology.kill:Nuking machines: ['smithi029', 'smithi121']
2022-02-07T21:48:43.560 INFO:teuthology.kill:2022-02-07 21:48:43,559.559 INFO:teuthology.nuke:targets:
2022-02-07T21:48:43.561 INFO:teuthology.kill:  smithi029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMpkbUOA22dY286YXBWwKR/ftitrXNJid+OsAvxXkRaOHUYD/0SWNq5WRy8WCF7alDwRQtVTt1DSbXdnFJ9Tqi/cI7vZmr9bCvgcSKbx4UmnHnha+SWG0c83VSH9JYViwAxUIiqKuXpcvYmc+1G/36Yq9iRmS71pE+eyRiywB7MSLetmhRELcEzDjkYtqNNtqaQpc2aTH6XkSMkhP81QCpi3ErsEpvLHR8iy8R07l66DrsgjwIlwXoMuyP+KiNj7kPAlceLI/iTv3AtN6j3YNPWb2/o4ihwp9BvCR63Lj0v44A1tiVwPmxakNJqPGZpV+z/snEZesGx1uyFOEkc+sc8WdetOAylkSdyII+hmv9sr3Y9uCKtY/YdcBaU8T/yjROakLVtrZWkwRUnguVV688BJGgdjmuomN4e72IiXJsXLnBcYnOQyUVfjU+ReU3H3KrDWQW8E+3J/NhhqzxmunKbXxpSbc9giK6jjwWwT2nMtFffBevx83h9oEL0pR1yxE=
2022-02-07T21:48:43.561 INFO:teuthology.kill:  smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwg9wxi0ZulvBfrs3V6Lp1OBaravit6jeTsXgHMBz1mJamLKiZNzdHY30PMWI8QJ3fMWbL7niFbABfer8idwuoiEmUPR91W0cg01VPNZNivX+pA3FSA4WI38L5JjcJYQJtZbQrBSv+seMADLklPegMpqACDkykm0TQ0DLwcBXPgt8SV2wXgece/GMNzD5fzGzzrw9ZmE26+vdRJQTheHAt+GIFMol1qXN0o5EQHFbeLBwrA2KmtXlzotCFCgrPTmR0MsOXLJ3s1wweI88bqV556CCHrluBNCmszFP3/ynprjEdIqPDc9pVaHkthDjesKAQyQmPK2GyPK6bgoQb/5pUaOtTjN3Q4YoXxx3jCZ3SB3Qmjft03a6ukbGVNvJC+8U6gHXfFtXDVD50w1V6wKRtF8EjSXESlMD3/Z08vF3gLqKVOX++0Y/h86uW5xLyJZWAe/FydlxkeIuVV1r31zbXH2M5onpnwWJYW02iTFGjpjQl9QtOzSasDvgaqGoPIcc=
2022-02-07T21:48:43.596 INFO:teuthology.kill:2022-02-07 21:48:43,596.596 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-07T21:48:43.602 INFO:teuthology.kill:2022-02-07 21:48:43,602.602 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-07T21:48:43.680 INFO:teuthology.kill:2022-02-07 21:48:43,678.678 INFO:teuthology.orchestra.console:Power off smithi121
2022-02-07T21:48:43.696 INFO:teuthology.kill:2022-02-07 21:48:43,696.696 INFO:teuthology.orchestra.console:Power off smithi029
2022-02-07T21:48:52.079 INFO:teuthology.kill:2022-02-07 21:48:52,078.078 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2022-02-07T21:48:52.225 INFO:teuthology.kill:2022-02-07 21:48:52,224.224 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com
2022-02-07T21:48:56.173 INFO:teuthology.kill:2022-02-07 21:48:56,172.172 INFO:teuthology.orchestra.console:Power off for smithi121 completed
2022-02-07T21:48:56.320 INFO:teuthology.kill:2022-02-07 21:48:56,320.320 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com
2022-02-07T21:50:56.542 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-07T21:50:56.658 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi029 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6666938
2022-02-07T21:50:56.659 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi121 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6666938