2023-06-14T21:26:07.524 INFO:root:teuthology version: 0.0.1.dev124+g961f4fb 2023-06-14T21:26:32.709 INFO:teuthology.lock.ops:Start node 'smithi070.front.sepia.ceph.com' reimaging 2023-06-14T21:26:32.710 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: reset os type and version on server 2023-06-14T21:26:32.710 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2023-06-14T21:26:34.003 INFO:teuthology.lock.ops:Start node 'smithi102.front.sepia.ceph.com' reimaging 2023-06-14T21:26:34.004 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: reset os type and version on server 2023-06-14T21:26:34.004 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2023-06-14T21:26:58.066 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of centos 8.stream 2023-06-14T21:26:58.124 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of centos 8.stream 2023-06-14T21:27:12.427 INFO:teuthology.orchestra.console:Power off smithi070 2023-06-14T21:27:12.448 INFO:teuthology.orchestra.console:Power off smithi102 2023-06-14T21:27:12.584 INFO:teuthology.orchestra.console:Power off for smithi070 completed 2023-06-14T21:27:12.594 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2023-06-14T21:27:12.686 INFO:teuthology.orchestra.console:Power on smithi070 2023-06-14T21:27:12.697 INFO:teuthology.orchestra.console:Power on smithi102 2023-06-14T21:27:48.500 INFO:teuthology.orchestra.console:Power on for smithi102 completed 2023-06-14T21:27:48.601 INFO:teuthology.provision.fog.smithi102:Waiting for deploy to finish 2023-06-14T21:27:58.835 INFO:teuthology.orchestra.console:Power on for smithi070 completed 2023-06-14T21:27:58.936 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish 2023-06-14T21:28:01.808 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102 2023-06-14T21:28:04.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-06-14T21:28:15.698 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102 2023-06-14T21:28:18.768 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-06-14T21:28:24.752 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102 2023-06-14T21:28:35.757 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-06-14T21:28:54.128 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102 2023-06-14T21:29:05.137 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-06-14T21:29:05.491 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102 2023-06-14T21:29:21.221 ERROR:teuthology.orchestra.connection:Error authenticating with smithi070.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:29:21.226 ERROR:teuthology.orchestra.connection:Error authenticating with smithi102.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:29:32.302 ERROR:teuthology.orchestra.connection:Error authenticating with smithi070.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:29:32.308 ERROR:teuthology.orchestra.connection:Error authenticating with smithi102.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:29:41.621 ERROR:teuthology.orchestra.connection:Error authenticating with smithi070.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:29:41.624 ERROR:teuthology.orchestra.connection:Error authenticating with smithi102.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:29:52.703 ERROR:teuthology.orchestra.connection:Error authenticating with smithi070.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:29:52.746 ERROR:teuthology.orchestra.connection:Error authenticating with smithi102.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:30:03.780 ERROR:teuthology.orchestra.connection:Error authenticating with smithi070.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:30:03.997 ERROR:teuthology.orchestra.connection:Error authenticating with smithi102.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:30:04.893 ERROR:teuthology.orchestra.connection:Error authenticating with smithi070.front.sepia.ceph.com: Authentication failed. 2023-06-14T21:31:10.002 WARNING:teuthology.provision.fog:timed out 2023-06-14T21:31:10.004 WARNING:teuthology.provision.fog:timed out 2023-06-14T21:31:34.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-06-14T21:31:34.385 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102 2023-06-14T21:31:50.608 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-14T21:31:50.720 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-14T21:32:10.548 INFO:teuthology.provision.fog.smithi102:Node is ready 2023-06-14T21:32:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102.front.sepia.ceph.com 2023-06-14T21:32:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102 smithi102.front.sepia.ceph.com smithi102 2023-06-14T21:32:10.724 INFO:teuthology.provision.fog.smithi070:Node is ready 2023-06-14T21:32:10.748 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070.front.sepia.ceph.com 2023-06-14T21:32:10.808 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70 smithi070.front.sepia.ceph.com smithi070 2023-06-14T21:32:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2023-06-14T21:32:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8" 2023-06-14T21:32:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2023-06-14T21:32:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2023-06-14T21:32:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8" 2023-06-14T21:32:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8" 2023-06-14T21:32:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-14T21:32:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2023-06-14T21:32:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-14T21:32:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2023-06-14T21:32:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-14T21:32:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-14T21:32:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-14T21:32:11.047 INFO:teuthology.provision.fog.smithi102:Deploy complete! 2023-06-14T21:32:11.228 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2023-06-14T21:32:11.228 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8" 2023-06-14T21:32:11.229 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2023-06-14T21:32:11.229 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2023-06-14T21:32:11.229 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8" 2023-06-14T21:32:11.230 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8" 2023-06-14T21:32:11.230 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-14T21:32:11.230 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2023-06-14T21:32:11.231 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-14T21:32:11.231 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2023-06-14T21:32:11.231 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-14T21:32:11.232 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-14T21:32:11.232 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-14T21:32:11.233 INFO:teuthology.provision.fog.smithi070:Deploy complete! 2023-06-14T21:32:11.915 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com 2023-06-14T21:32:11.916 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-14T21:32:11.939 INFO:teuthology.lock.ops:Checking smithi102.front.sepia.ceph.com 2023-06-14T21:32:11.940 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-14T21:32:11.966 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server 2023-06-14T21:32:11.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-06-14T21:32:12.120 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2023-06-14T21:32:12.179 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2023-06-14T21:32:12.180 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8" 2023-06-14T21:32:12.180 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2023-06-14T21:32:12.180 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2023-06-14T21:32:12.180 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8" 2023-06-14T21:32:12.180 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8" 2023-06-14T21:32:12.181 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-14T21:32:12.181 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2023-06-14T21:32:12.181 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-14T21:32:12.181 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2023-06-14T21:32:12.181 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-14T21:32:12.181 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-14T21:32:12.181 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-14T21:32:12.182 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2023-06-14T21:32:12.206 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: set os type and version on server 2023-06-14T21:32:12.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2023-06-14T21:32:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2023-06-14T21:32:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2023-06-14T21:32:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8" 2023-06-14T21:32:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2023-06-14T21:32:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2023-06-14T21:32:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8" 2023-06-14T21:32:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8" 2023-06-14T21:32:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-14T21:32:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2023-06-14T21:32:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-14T21:32:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2023-06-14T21:32:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-14T21:32:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-14T21:32:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-14T21:32:12.467 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2023-06-14T21:32:12.595 INFO:teuthology.dispatcher.supervisor:Running job 7304587 2023-06-14T21:32:12.596 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_961f4fb51318c373681de5844aadbc6dc0e58abc/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2023-06-14_21:15:41-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi/7304587 --name adking-2023-06-14_21:15:41-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/adking-2023-06-14_21:15:41-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi/7304587/orig.config.yaml 2023-06-14T21:32:12.602 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2023-06-14_21:15:41-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi/7304587 2023-06-14T21:32:12.603 INFO:teuthology.dispatcher.supervisor:Job PID: 26232 2023-06-14T21:32:12.603 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-06-14T21:44:40.053 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-06-14T21:44:50.217 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-06-14T21:44:50.217 INFO:teuthology.nuke:Checking targets against current locks 2023-06-14T21:44:50.301 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-14T21:44:50.320 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-14T21:44:50.378 INFO:teuthology.orchestra.console:Power off smithi070 2023-06-14T21:44:50.410 INFO:teuthology.orchestra.console:Power off smithi102 2023-06-14T21:45:00.625 INFO:teuthology.orchestra.console:Power off for smithi070 completed 2023-06-14T21:45:10.798 INFO:teuthology.lock.ops:unlocked: smithi070.front.sepia.ceph.com 2023-06-14T21:45:21.193 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2023-06-14T21:45:31.363 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com