2023-06-08T15:11:20.369 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd 2023-06-08T15:11:20.525 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging 2023-06-08T15:11:20.526 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server 2023-06-08T15:11:20.527 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-06-08T15:11:20.551 INFO:teuthology.lock.ops:Start node 'smithi115.front.sepia.ceph.com' reimaging 2023-06-08T15:11:20.552 INFO:teuthology.lock.ops:Updating [smithi115.front.sepia.ceph.com]: reset os type and version on server 2023-06-08T15:11:20.552 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2023-06-08T15:11:20.731 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of rhel 8.4 2023-06-08T15:11:20.741 INFO:teuthology.provision.fog.smithi115:Scheduling deploy of rhel 8.4 2023-06-08T15:11:21.415 INFO:teuthology.orchestra.console:Power off smithi050 2023-06-08T15:11:21.431 INFO:teuthology.orchestra.console:Power off smithi115 2023-06-08T15:11:21.565 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2023-06-08T15:11:21.666 INFO:teuthology.orchestra.console:Power on smithi050 2023-06-08T15:11:29.803 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2023-06-08T15:11:29.905 INFO:teuthology.orchestra.console:Power on smithi115 2023-06-08T15:11:30.035 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2023-06-08T15:11:30.136 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2023-06-08T15:11:38.271 INFO:teuthology.orchestra.console:Power on for smithi115 completed 2023-06-08T15:11:38.372 INFO:teuthology.provision.fog.smithi115:Waiting for deploy to finish 2023-06-08T15:16:00.048 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-06-08T15:16:06.964 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-06-08T15:16:09.108 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-06-08T15:16:16.048 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-06-08T15:16:18.192 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-06-08T15:16:25.108 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-06-08T15:16:27.248 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-06-08T15:16:34.192 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-06-08T15:16:36.308 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-06-08T15:16:43.248 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-06-08T15:16:45.392 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-06-08T15:16:52.308 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-06-08T15:16:54.448 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-06-08T15:17:58.371 WARNING:teuthology.provision.fog:timed out 2023-06-08T15:18:00.458 WARNING:teuthology.provision.fog:timed out 2023-06-08T15:18:04.676 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-08T15:18:04.895 INFO:teuthology.provision.fog.smithi050:Node is ready 2023-06-08T15:18:04.951 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2023-06-08T15:18:05.007 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2023-06-08T15:18:05.402 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Red Hat Enterprise Linux" 2023-06-08T15:18:05.403 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8.4 (Ootpa)" 2023-06-08T15:18:05.403 INFO:teuthology.orchestra.run.smithi050.stdout:ID="rhel" 2023-06-08T15:18:05.403 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="fedora" 2023-06-08T15:18:05.403 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8.4" 2023-06-08T15:18:05.403 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8" 2023-06-08T15:18:05.403 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-08T15:18:05.403 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2023-06-08T15:18:05.404 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-08T15:18:05.404 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.redhat.com/" 2023-06-08T15:18:05.404 INFO:teuthology.orchestra.run.smithi050.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-08T15:18:05.404 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-08T15:18:05.404 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-06-08T15:18:05.404 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-08T15:18:05.404 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-08T15:18:05.405 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-08T15:18:05.405 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-08T15:18:05.405 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2023-06-08T15:18:06.734 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-08T15:18:06.965 INFO:teuthology.provision.fog.smithi115:Node is ready 2023-06-08T15:18:07.020 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115.front.sepia.ceph.com 2023-06-08T15:18:07.076 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115 smithi115.front.sepia.ceph.com smithi115 2023-06-08T15:18:07.452 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Red Hat Enterprise Linux" 2023-06-08T15:18:07.452 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="8.4 (Ootpa)" 2023-06-08T15:18:07.452 INFO:teuthology.orchestra.run.smithi115.stdout:ID="rhel" 2023-06-08T15:18:07.452 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="fedora" 2023-06-08T15:18:07.453 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="8.4" 2023-06-08T15:18:07.453 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el8" 2023-06-08T15:18:07.453 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-08T15:18:07.453 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2023-06-08T15:18:07.453 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-08T15:18:07.453 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.redhat.com/" 2023-06-08T15:18:07.453 INFO:teuthology.orchestra.run.smithi115.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-08T15:18:07.454 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-08T15:18:07.454 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-06-08T15:18:07.454 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-08T15:18:07.454 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-08T15:18:07.454 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-08T15:18:07.454 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-08T15:18:07.455 INFO:teuthology.provision.fog.smithi115:Deploy complete! 2023-06-08T15:18:07.621 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2023-06-08T15:18:07.622 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-08T15:18:07.649 INFO:teuthology.lock.ops:Checking smithi115.front.sepia.ceph.com 2023-06-08T15:18:07.650 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-08T15:18:07.676 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2023-06-08T15:18:07.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-06-08T15:18:07.870 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2023-06-08T15:18:07.927 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Red Hat Enterprise Linux" 2023-06-08T15:18:07.928 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8.4 (Ootpa)" 2023-06-08T15:18:07.928 INFO:teuthology.orchestra.run.smithi050.stdout:ID="rhel" 2023-06-08T15:18:07.928 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="fedora" 2023-06-08T15:18:07.928 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8.4" 2023-06-08T15:18:07.929 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8" 2023-06-08T15:18:07.929 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-08T15:18:07.929 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2023-06-08T15:18:07.929 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-08T15:18:07.929 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.redhat.com/" 2023-06-08T15:18:07.929 INFO:teuthology.orchestra.run.smithi050.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-08T15:18:07.929 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-08T15:18:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-06-08T15:18:07.930 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-08T15:18:07.930 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-08T15:18:07.930 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-08T15:18:07.932 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-08T15:18:07.933 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-06-08T15:18:07.958 INFO:teuthology.lock.ops:Updating [smithi115.front.sepia.ceph.com]: set os type and version on server 2023-06-08T15:18:07.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-06-08T15:18:08.149 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2023-06-08T15:18:08.206 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Red Hat Enterprise Linux" 2023-06-08T15:18:08.207 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="8.4 (Ootpa)" 2023-06-08T15:18:08.207 INFO:teuthology.orchestra.run.smithi115.stdout:ID="rhel" 2023-06-08T15:18:08.207 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="fedora" 2023-06-08T15:18:08.207 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="8.4" 2023-06-08T15:18:08.207 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el8" 2023-06-08T15:18:08.207 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-08T15:18:08.208 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2023-06-08T15:18:08.208 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-08T15:18:08.208 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.redhat.com/" 2023-06-08T15:18:08.208 INFO:teuthology.orchestra.run.smithi115.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-08T15:18:08.208 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-08T15:18:08.208 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-06-08T15:18:08.208 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-08T15:18:08.209 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-08T15:18:08.209 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-08T15:18:08.209 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-08T15:18:08.209 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2023-06-08T15:18:08.344 INFO:teuthology.dispatcher.supervisor:Running job 7298767 2023-06-08T15:18:08.344 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-08_15:10:10-rgw-wip-yuri8-testing-2023-06-07-1323-quincy-distro-default-smithi/7298767 --name yuriw-2023-06-08_15:10:10-rgw-wip-yuri8-testing-2023-06-07-1323-quincy-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{rhel_8} tasks/test_multi} -- /home/teuthworker/archive/yuriw-2023-06-08_15:10:10-rgw-wip-yuri8-testing-2023-06-07-1323-quincy-distro-default-smithi/7298767/orig.config.yaml 2023-06-08T15:18:08.349 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-06-08_15:10:10-rgw-wip-yuri8-testing-2023-06-07-1323-quincy-distro-default-smithi/7298767 2023-06-08T15:18:08.350 INFO:teuthology.dispatcher.supervisor:Job PID: 22507 2023-06-08T15:18:08.350 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-06-08T16:26:14.565 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-06-08T16:26:14.753 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-06-08T16:26:14.754 INFO:teuthology.nuke:Checking targets against current locks 2023-06-08T16:26:14.808 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-08T16:26:14.834 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-08T16:26:14.883 INFO:teuthology.orchestra.console:Power off smithi050 2023-06-08T16:26:14.931 INFO:teuthology.orchestra.console:Power off smithi115 2023-06-08T16:26:15.041 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2023-06-08T16:26:15.074 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2023-06-08T16:26:15.197 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com 2023-06-08T16:26:15.231 INFO:teuthology.lock.ops:unlocked: smithi115.front.sepia.ceph.com