2023-12-10T05:58:59.065 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-10T05:58:59.065 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T05:58:59.193 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f415f3bd550>, signals=[15]) 2023-12-10T05:58:59.193 INFO:teuthology.lock.ops:Start node 'smithi155.front.sepia.ceph.com' reimaging 2023-12-10T05:58:59.193 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: reset os type and version on server 2023-12-10T05:58:59.194 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-12-10T05:58:59.213 INFO:teuthology.lock.ops:Start node 'smithi183.front.sepia.ceph.com' reimaging 2023-12-10T05:58:59.213 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: reset os type and version on server 2023-12-10T05:58:59.213 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-12-10T05:58:59.325 DEBUG:teuthology.provision.fog.smithi155:Requesting image smithi_centos_9.stream (ID 10) 2023-12-10T05:58:59.328 DEBUG:teuthology.provision.fog.smithi183:Requesting image smithi_centos_9.stream (ID 10) 2023-12-10T05:58:59.372 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of centos 9.stream 2023-12-10T05:58:59.375 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of centos 9.stream 2023-12-10T05:58:59.661 INFO:teuthology.orchestra.console.smithi183:Power off 2023-12-10T05:58:59.661 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T05:58:59.677 INFO:teuthology.orchestra.console.smithi155:Power off 2023-12-10T05:58:59.677 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T05:58:59.684 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-12-10T05:58:59.685 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:58:59.698 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2023-12-10T05:58:59.698 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:58:59.706 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-12-10T05:58:59.720 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-12-10T05:59:03.711 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:03.722 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:03.820 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-12-10T05:59:03.830 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-12-10T05:59:07.825 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:07.832 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:07.933 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-12-10T05:59:07.940 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-12-10T05:59:08.035 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-12-10T05:59:08.135 INFO:teuthology.orchestra.console.smithi183:Power on 2023-12-10T05:59:08.135 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-10T05:59:08.156 DEBUG:teuthology.orchestra.console.smithi183:power on output: Chassis Power Control: Up/On 2023-12-10T05:59:08.156 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:08.177 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-12-10T05:59:11.945 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:12.053 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-12-10T05:59:12.154 INFO:teuthology.orchestra.console.smithi155:Power off completed 2023-12-10T05:59:12.177 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:12.255 INFO:teuthology.orchestra.console.smithi155:Power on 2023-12-10T05:59:12.255 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-10T05:59:12.274 DEBUG:teuthology.orchestra.console.smithi155:power on output: Chassis Power Control: Up/On 2023-12-10T05:59:12.274 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:12.286 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-12-10T05:59:12.295 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-12-10T05:59:16.291 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:16.297 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T05:59:16.399 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-12-10T05:59:16.404 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-12-10T05:59:16.500 INFO:teuthology.orchestra.console.smithi183:Power on completed 2023-12-10T05:59:16.505 INFO:teuthology.orchestra.console.smithi155:Power on completed 2023-12-10T05:59:16.601 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish 2023-12-10T05:59:16.606 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2023-12-10T06:01:51.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:01:54.274 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:02:00.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:02:03.362 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:02:09.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:02:12.422 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:02:18.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:02:21.506 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:02:27.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:02:30.562 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:02:36.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:02:39.650 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:02:45.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:03:04.162 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:03:10.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:03:13.218 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:03:19.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:03:22.306 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-12-10T06:03:28.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:03:28.659 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-10T06:03:28.660 DEBUG:teuthology.orchestra.run.smithi183:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-10T06:03:43.977 INFO:teuthology.provision.fog.smithi183:Node is ready 2023-12-10T06:03:43.978 DEBUG:teuthology.orchestra.run.smithi183:> hostname 2023-12-10T06:03:43.995 INFO:teuthology.orchestra.run.smithi183.stdout:smithi046 2023-12-10T06:03:43.995 DEBUG:teuthology.orchestra.run.smithi183:> grep smithi046 /etc/hosts 2023-12-10T06:03:44.052 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2023-12-10T06:03:44.052 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183 2023-12-10T06:03:44.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi046/smithi183/g' /etc/hosts 2023-12-10T06:03:44.223 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/172.21.15.46/172.21.15.183/g' /etc/hosts 2023-12-10T06:03:44.293 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi046/smithi183/g' /etc/hostname 2023-12-10T06:03:44.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183 2023-12-10T06:03:44.429 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2023-12-10T06:03:44.484 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T06:03:44.485 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T06:03:44.485 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T06:03:44.485 INFO:teuthology.provision.fog.smithi183:Deploy complete! 2023-12-10T06:03:44.486 DEBUG:teuthology.parallel:result is None 2023-12-10T06:05:27.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:05:30.274 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:05:36.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:05:39.330 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:05:45.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:05:48.386 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:05:54.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:05:57.474 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:06:03.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:06:06.530 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:06:12.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:06:15.618 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:06:21.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:06:24.674 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:06:30.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:06:49.190 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:06:55.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:06:58.274 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:07:04.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:07:07.330 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-12-10T06:07:13.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:07:13.522 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-10T06:07:13.522 DEBUG:teuthology.orchestra.run.smithi155:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-10T06:07:29.021 INFO:teuthology.provision.fog.smithi155:Node is ready 2023-12-10T06:07:29.021 DEBUG:teuthology.orchestra.run.smithi155:> hostname 2023-12-10T06:07:29.038 INFO:teuthology.orchestra.run.smithi155.stdout:smithi046 2023-12-10T06:07:29.039 DEBUG:teuthology.orchestra.run.smithi155:> grep smithi046 /etc/hosts 2023-12-10T06:07:29.095 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2023-12-10T06:07:29.096 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname smithi155 2023-12-10T06:07:29.257 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi046/smithi155/g' /etc/hosts 2023-12-10T06:07:29.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/172.21.15.46/172.21.15.155/g' /etc/hosts 2023-12-10T06:07:29.357 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi046/smithi155/g' /etc/hostname 2023-12-10T06:07:29.438 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname smithi155 2023-12-10T06:07:29.506 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-12-10T06:07:29.560 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2023-12-10T06:07:29.560 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2023-12-10T06:07:29.561 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2023-12-10T06:07:29.561 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2023-12-10T06:07:29.561 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2023-12-10T06:07:29.561 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2023-12-10T06:07:29.561 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T06:07:29.562 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-12-10T06:07:29.562 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2023-12-10T06:07:29.562 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T06:07:29.562 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2023-12-10T06:07:29.562 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T06:07:29.563 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T06:07:29.563 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T06:07:29.563 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2023-12-10T06:07:29.564 DEBUG:teuthology.parallel:result is None 2023-12-10T06:07:29.733 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2023-12-10T06:07:29.733 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-10T06:07:29.756 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com 2023-12-10T06:07:29.756 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-10T06:07:29.774 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2023-12-10T06:07:29.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-12-10T06:07:29.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:07:29.858 DEBUG:teuthology.orchestra.run.smithi155:> true 2023-12-10T06:07:29.924 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-12-10T06:07:29.924 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2023-12-10T06:07:29.978 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2023-12-10T06:07:29.978 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-12-10T06:07:30.032 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T06:07:30.033 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T06:07:30.034 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T06:07:30.034 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-12-10T06:07:30.056 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server 2023-12-10T06:07:30.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-12-10T06:07:30.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T06:07:30.141 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-12-10T06:07:30.312 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-12-10T06:07:30.312 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2023-12-10T06:07:30.366 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2023-12-10T06:07:30.366 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-12-10T06:07:30.420 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2023-12-10T06:07:30.420 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T06:07:30.421 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T06:07:30.422 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T06:07:30.422 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-12-10T06:07:30.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:07:30.525 INFO:teuthology.dispatcher.supervisor:Running job 7485728 2023-12-10T06:07:30.525 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2023-12-10_05:57:50-orch:cephadm-wip-adk4-centos-9-runc-2023-12-09-2102-distro-default-smithi/7485728 --name adking-2023-12-10_05:57:50-orch:cephadm-wip-adk4-centos-9-runc-2023-12-09-2102-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/adking-2023-12-10_05:57:50-orch:cephadm-wip-adk4-centos-9-runc-2023-12-09-2102-distro-default-smithi/7485728/orig.config.yaml 2023-12-10T06:07:30.528 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2023-12-10_05:57:50-orch:cephadm-wip-adk4-centos-9-runc-2023-12-09-2102-distro-default-smithi/7485728 2023-12-10T06:07:30.528 INFO:teuthology.dispatcher.supervisor:Job PID: 60078 2023-12-10T06:07:30.528 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-10T06:09:30.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:11:30.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:13:30.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:15:31.168 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:17:31.348 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:19:31.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:21:31.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:23:31.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:25:32.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:27:32.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:29:32.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:31:32.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:33:32.664 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:35:32.823 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:37:32.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:39:33.230 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:41:33.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:43:33.577 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:45:33.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T06:45:33.875 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-10T06:45:33.986 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-10T06:45:33.986 INFO:teuthology.nuke:Checking targets against current locks 2023-12-10T06:45:34.026 DEBUG:teuthology.nuke:shortname: smithi155 2023-12-10T06:45:34.027 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-10T06:45:34.041 DEBUG:teuthology.nuke:shortname: smithi183 2023-12-10T06:45:34.041 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-10T06:45:34.045 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-12-10_05:57:50-orch:cephadm-wip-adk4-centos-9-runc-2023-12-09-2102-distro-default-smithi/7485728', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-10 05:58:58.296835', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCmk+sL0TBNZYc7CFfFMVAp9SnniANp0nSs1L7KV0S3CtNYXqR+vY6kI115qyt6mGbO57r/aXsUbC33ClAr9bkM='} 2023-12-10T06:45:34.059 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-12-10_05:57:50-orch:cephadm-wip-adk4-centos-9-runc-2023-12-09-2102-distro-default-smithi/7485728', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-10 05:58:58.295053', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLC3dAdNCkeT6qfK3B6QuKyanmx499X0idJ47Mix/Dy022FVoR+GmOAlxmidr5I21A8gRNEgZc8uIUOW5oHz6jc='} 2023-12-10T06:45:34.083 INFO:teuthology.orchestra.console.smithi155:Power off 2023-12-10T06:45:34.083 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T06:45:34.105 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2023-12-10T06:45:34.105 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T06:45:34.123 INFO:teuthology.orchestra.console.smithi183:Power off 2023-12-10T06:45:34.123 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T06:45:34.130 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-12-10T06:45:34.142 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-12-10T06:45:34.142 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T06:45:34.161 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-12-10T06:45:34.232 INFO:teuthology.orchestra.console.smithi155:Power off completed 2023-12-10T06:45:34.262 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-12-10T06:45:34.382 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com 2023-12-10T06:45:34.407 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com