2023-10-12T22:10:29.658 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-12T22:10:29.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:10:29.801 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7fcbcd4670>, signals=[15]) 2023-10-12T22:10:29.802 INFO:teuthology.lock.ops:Start node 'smithi131.front.sepia.ceph.com' reimaging 2023-10-12T22:10:29.803 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: reset os type and version on server 2023-10-12T22:10:29.803 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2023-10-12T22:10:29.820 INFO:teuthology.lock.ops:Start node 'smithi195.front.sepia.ceph.com' reimaging 2023-10-12T22:10:29.821 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: reset os type and version on server 2023-10-12T22:10:29.821 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2023-10-12T22:10:29.822 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /host {"name": "smithi131"} 2023-10-12T22:10:29.845 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /host {"name": "smithi195"} 2023-10-12T22:10:29.903 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-12T22:10:29.908 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-12T22:10:29.950 DEBUG:teuthology.provision.fog.smithi131:Requesting image smithi_centos_8.stream (ID 3) 2023-10-12T22:10:29.950 DEBUG:teuthology.provision.fog.smithi131:FOG request: PUT /host/236 {"imageID": 3} 2023-10-12T22:10:29.951 DEBUG:teuthology.provision.fog.smithi195:Requesting image smithi_centos_8.stream (ID 3) 2023-10-12T22:10:29.951 DEBUG:teuthology.provision.fog.smithi195:FOG request: PUT /host/300 {"imageID": 3} 2023-10-12T22:10:30.001 INFO:teuthology.provision.fog.smithi131:Scheduling deploy of centos 8.stream 2023-10-12T22:10:30.002 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:10:30.005 INFO:teuthology.provision.fog.smithi195:Scheduling deploy of centos 8.stream 2023-10-12T22:10:30.005 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:10:30.065 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /tasktype {"name": "deploy"} 2023-10-12T22:10:30.070 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /tasktype {"name": "deploy"} 2023-10-12T22:10:30.106 DEBUG:teuthology.provision.fog.smithi195:FOG request: POST /host/300/task {"taskTypeID": 1} 2023-10-12T22:10:30.110 DEBUG:teuthology.provision.fog.smithi131:FOG request: POST /host/236/task {"taskTypeID": 1} 2023-10-12T22:10:30.154 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:10:30.159 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:10:30.253 INFO:teuthology.orchestra.console.smithi131:Power off 2023-10-12T22:10:30.254 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-12T22:10:30.266 INFO:teuthology.orchestra.console.smithi195:Power off 2023-10-12T22:10:30.267 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-12T22:10:30.276 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2023-10-12T22:10:30.276 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:30.289 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2023-10-12T22:10:30.289 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:30.298 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-10-12T22:10:30.313 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2023-10-12T22:10:34.302 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:34.315 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:34.412 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-10-12T22:10:34.424 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2023-10-12T22:10:38.414 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:38.424 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:38.524 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-10-12T22:10:38.534 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-10-12T22:10:38.635 INFO:teuthology.orchestra.console.smithi195:Power off completed 2023-10-12T22:10:38.735 INFO:teuthology.orchestra.console.smithi195:Power on 2023-10-12T22:10:38.736 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-12T22:10:38.779 DEBUG:teuthology.orchestra.console.smithi195:power on output: Chassis Power Control: Up/On 2023-10-12T22:10:38.780 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:38.816 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-10-12T22:10:42.526 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:42.636 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-10-12T22:10:42.737 INFO:teuthology.orchestra.console.smithi131:Power off completed 2023-10-12T22:10:42.817 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:42.838 INFO:teuthology.orchestra.console.smithi131:Power on 2023-10-12T22:10:42.839 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-12T22:10:42.859 DEBUG:teuthology.orchestra.console.smithi131:power on output: Chassis Power Control: Up/On 2023-10-12T22:10:42.860 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:42.888 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-10-12T22:10:42.927 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-10-12T22:10:46.890 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:46.929 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:10:46.999 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-10-12T22:10:47.039 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2023-10-12T22:10:47.100 INFO:teuthology.orchestra.console.smithi131:Power on completed 2023-10-12T22:10:47.140 INFO:teuthology.orchestra.console.smithi195:Power on completed 2023-10-12T22:10:47.202 INFO:teuthology.provision.fog.smithi131:Waiting for deploy to finish 2023-10-12T22:10:47.202 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:10:47.241 INFO:teuthology.provision.fog.smithi195:Waiting for deploy to finish 2023-10-12T22:10:47.241 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:11:02.282 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:11:02.305 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:11:17.374 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:11:17.389 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:11:32.462 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:11:32.464 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:11:47.554 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:11:47.556 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:12:02.634 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:12:02.636 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:12:17.718 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:12:17.720 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:12:32.807 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:12:32.809 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:12:47.902 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:12:47.904 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:13:02.998 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:13:03.000 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:13:18.094 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:13:18.096 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-12T22:13:18.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:13:33.186 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-10-12T22:13:33.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:13:36.510 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-12T22:13:42.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:13:45.594 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-12T22:13:51.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:13:51.614 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-10-12T22:13:54.654 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-12T22:13:57.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:14:00.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:14:00.698 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-10-12T22:14:03.738 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-12T22:14:06.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:14:09.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:14:09.754 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-10-12T22:14:12.798 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-12T22:14:15.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:14:18.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:14:18.810 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-10-12T22:14:24.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:14:43.322 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-10-12T22:14:49.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:14:50.469 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-12T22:14:50.470 DEBUG:teuthology.orchestra.run.smithi131:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-12T22:14:52.383 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-10-12T22:14:58.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:15:00.266 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-12T22:15:00.266 DEBUG:teuthology.orchestra.run.smithi195:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-12T22:15:15.277 INFO:teuthology.provision.fog.smithi131:Node is ready 2023-10-12T22:15:15.277 DEBUG:teuthology.orchestra.run.smithi131:> hostname 2023-10-12T22:15:15.301 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131.front.sepia.ceph.com 2023-10-12T22:15:15.302 DEBUG:teuthology.orchestra.run.smithi131:> grep smithi131.front.sepia.ceph.com /etc/hosts 2023-10-12T22:15:15.361 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131 smithi131.front.sepia.ceph.com smithi131 2023-10-12T22:15:15.362 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131 2023-10-12T22:15:15.485 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hosts 2023-10-12T22:15:15.517 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/172.21.15.131/172.21.15.131/g' /etc/hosts 2023-10-12T22:15:15.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hostname 2023-10-12T22:15:15.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131 2023-10-12T22:15:15.725 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2023-10-12T22:15:15.784 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2023-10-12T22:15:15.784 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8" 2023-10-12T22:15:15.785 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2023-10-12T22:15:15.785 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2023-10-12T22:15:15.785 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8" 2023-10-12T22:15:15.785 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2023-10-12T22:15:15.785 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-12T22:15:15.786 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2023-10-12T22:15:15.786 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-12T22:15:15.786 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2023-10-12T22:15:15.786 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-12T22:15:15.786 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-12T22:15:15.787 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-12T22:15:15.787 INFO:teuthology.provision.fog.smithi131:Deploy complete! 2023-10-12T22:15:15.789 DEBUG:teuthology.parallel:result is None 2023-10-12T22:15:25.983 INFO:teuthology.provision.fog.smithi195:Node is ready 2023-10-12T22:15:25.984 DEBUG:teuthology.orchestra.run.smithi195:> hostname 2023-10-12T22:15:26.008 INFO:teuthology.orchestra.run.smithi195.stdout:smithi195.front.sepia.ceph.com 2023-10-12T22:15:26.008 DEBUG:teuthology.orchestra.run.smithi195:> grep smithi195.front.sepia.ceph.com /etc/hosts 2023-10-12T22:15:26.069 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.15.195 smithi195.front.sepia.ceph.com smithi195 2023-10-12T22:15:26.069 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195 2023-10-12T22:15:26.190 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hosts 2023-10-12T22:15:26.220 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/172.21.15.195/172.21.15.195/g' /etc/hosts 2023-10-12T22:15:26.290 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hostname 2023-10-12T22:15:26.359 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195 2023-10-12T22:15:26.427 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2023-10-12T22:15:26.487 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2023-10-12T22:15:26.487 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="8" 2023-10-12T22:15:26.487 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2023-10-12T22:15:26.488 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2023-10-12T22:15:26.488 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="8" 2023-10-12T22:15:26.488 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el8" 2023-10-12T22:15:26.488 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-12T22:15:26.488 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2023-10-12T22:15:26.489 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-12T22:15:26.489 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2023-10-12T22:15:26.489 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-12T22:15:26.489 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-12T22:15:26.489 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-12T22:15:26.490 INFO:teuthology.provision.fog.smithi195:Deploy complete! 2023-10-12T22:15:26.490 DEBUG:teuthology.parallel:result is None 2023-10-12T22:15:26.669 INFO:teuthology.lock.ops:Checking smithi131.front.sepia.ceph.com 2023-10-12T22:15:26.670 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-12T22:15:26.695 INFO:teuthology.lock.ops:Checking smithi195.front.sepia.ceph.com 2023-10-12T22:15:26.695 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-12T22:15:26.712 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: set os type and version on server 2023-10-12T22:15:26.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-10-12T22:15:26.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:15:26.782 DEBUG:teuthology.orchestra.run.smithi131:> true 2023-10-12T22:15:26.921 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-10-12T22:15:26.921 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2023-10-12T22:15:26.980 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2023-10-12T22:15:26.981 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2023-10-12T22:15:27.039 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2023-10-12T22:15:27.039 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8" 2023-10-12T22:15:27.039 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2023-10-12T22:15:27.040 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2023-10-12T22:15:27.040 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8" 2023-10-12T22:15:27.040 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2023-10-12T22:15:27.040 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-12T22:15:27.040 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2023-10-12T22:15:27.041 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-12T22:15:27.041 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2023-10-12T22:15:27.041 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-12T22:15:27.041 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-12T22:15:27.042 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-12T22:15:27.042 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2023-10-12T22:15:27.061 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: set os type and version on server 2023-10-12T22:15:27.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-10-12T22:15:27.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T22:15:27.129 DEBUG:teuthology.orchestra.run.smithi195:> true 2023-10-12T22:15:27.201 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-10-12T22:15:27.201 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2023-10-12T22:15:27.258 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2023-10-12T22:15:27.259 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2023-10-12T22:15:27.316 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2023-10-12T22:15:27.316 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="8" 2023-10-12T22:15:27.317 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2023-10-12T22:15:27.317 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2023-10-12T22:15:27.317 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="8" 2023-10-12T22:15:27.317 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el8" 2023-10-12T22:15:27.317 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-12T22:15:27.318 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2023-10-12T22:15:27.318 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-12T22:15:27.318 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2023-10-12T22:15:27.318 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-12T22:15:27.318 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-12T22:15:27.319 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-12T22:15:27.319 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2023-10-12T22:15:27.353 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:15:27.448 INFO:teuthology.dispatcher.supervisor:Running job 7423697 2023-10-12T22:15:27.448 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2023-10-12_22:09:36-orch:cephadm-wip-adk2-testing-2023-10-11-1434-pacific-distro-default-smithi/7423697 --name adking-2023-10-12_22:09:36-orch:cephadm-wip-adk2-testing-2023-10-11-1434-pacific-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/adking-2023-10-12_22:09:36-orch:cephadm-wip-adk2-testing-2023-10-11-1434-pacific-distro-default-smithi/7423697/orig.config.yaml 2023-10-12T22:15:27.451 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2023-10-12_22:09:36-orch:cephadm-wip-adk2-testing-2023-10-11-1434-pacific-distro-default-smithi/7423697 2023-10-12T22:15:27.452 INFO:teuthology.dispatcher.supervisor:Job PID: 224168 2023-10-12T22:15:27.453 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-12T22:17:27.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:19:27.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:21:27.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:23:28.079 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:25:28.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:27:28.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:29:28.503 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:31:28.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:33:28.823 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:35:28.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:37:29.160 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:39:29.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:41:29.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:43:29.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T22:43:29.722 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-12T22:43:29.962 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-12T22:43:29.963 INFO:teuthology.nuke:Checking targets against current locks 2023-10-12T22:43:30.017 DEBUG:teuthology.nuke:shortname: smithi131 2023-10-12T22:43:30.018 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-12T22:43:30.046 DEBUG:teuthology.nuke:shortname: smithi195 2023-10-12T22:43:30.046 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-12T22:43:30.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-10-12_22:09:36-orch:cephadm-wip-adk2-testing-2023-10-11-1434-pacific-distro-default-smithi/7423697', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-12 22:10:28.841347', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDFLy6TIRluuOU2mpRGEf+CPBYDhm02Kpe8KbVhgrHGHoHoTGdSrAkVbX8kXk+HL8vZuj+q8217tuUW8g3RFXjk='} 2023-10-12T22:43:30.078 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-10-12_22:09:36-orch:cephadm-wip-adk2-testing-2023-10-11-1434-pacific-distro-default-smithi/7423697', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-12 22:10:28.842732', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKqNVhDHWHENBC2MC/Z8H7Zf/Gxq9arVln7DmZtvPwArFWpMD0GrRioieAvw9z9E7fsHGA5ZVQC1nz/6g3XO/to='} 2023-10-12T22:43:30.110 INFO:teuthology.orchestra.console.smithi131:Power off 2023-10-12T22:43:30.110 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-12T22:43:30.139 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2023-10-12T22:43:30.140 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:43:30.158 INFO:teuthology.orchestra.console.smithi195:Power off 2023-10-12T22:43:30.159 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-12T22:43:30.170 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-10-12T22:43:30.184 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2023-10-12T22:43:30.184 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T22:43:30.213 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-10-12T22:43:30.273 INFO:teuthology.orchestra.console.smithi131:Power off completed 2023-10-12T22:43:30.315 INFO:teuthology.orchestra.console.smithi195:Power off completed 2023-10-12T22:43:30.432 INFO:teuthology.lock.ops:unlocked: smithi131.front.sepia.ceph.com 2023-10-12T22:43:30.463 INFO:teuthology.lock.ops:unlocked: smithi195.front.sepia.ceph.com