2023-11-13T11:40:41.054 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-13T11:40:41.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:40:41.215 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6e26703ee0>, signals=[15]) 2023-11-13T11:40:41.217 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging 2023-11-13T11:40:41.218 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server 2023-11-13T11:40:41.218 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-11-13T11:40:41.238 INFO:teuthology.lock.ops:Start node 'smithi195.front.sepia.ceph.com' reimaging 2023-11-13T11:40:41.239 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: reset os type and version on server 2023-11-13T11:40:41.239 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2023-11-13T11:40:41.241 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /host {"name": "smithi181"} 2023-11-13T11:40:41.264 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /host {"name": "smithi195"} 2023-11-13T11:40:41.317 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-13T11:40:41.334 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-13T11:40:41.358 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_centos_8.stream (ID 3) 2023-11-13T11:40:41.358 DEBUG:teuthology.provision.fog.smithi181:FOG request: PUT /host/286 {"imageID": 3} 2023-11-13T11:40:41.373 DEBUG:teuthology.provision.fog.smithi195:Requesting image smithi_centos_8.stream (ID 3) 2023-11-13T11:40:41.373 DEBUG:teuthology.provision.fog.smithi195:FOG request: PUT /host/300 {"imageID": 3} 2023-11-13T11:40:41.403 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 8.stream 2023-11-13T11:40:41.403 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:40:41.424 INFO:teuthology.provision.fog.smithi195:Scheduling deploy of centos 8.stream 2023-11-13T11:40:41.425 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:40:41.488 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /tasktype {"name": "deploy"} 2023-11-13T11:40:41.493 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /tasktype {"name": "deploy"} 2023-11-13T11:40:41.519 DEBUG:teuthology.provision.fog.smithi181:FOG request: POST /host/286/task {"taskTypeID": 1} 2023-11-13T11:40:41.530 DEBUG:teuthology.provision.fog.smithi195:FOG request: POST /host/300/task {"taskTypeID": 1} 2023-11-13T11:40:41.583 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:40:41.584 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:40:41.712 INFO:teuthology.orchestra.console.smithi195:Power off 2023-11-13T11:40:41.713 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T11:40:41.728 INFO:teuthology.orchestra.console.smithi181:Power off 2023-11-13T11:40:41.728 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T11:40:41.738 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2023-11-13T11:40:41.738 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:41.751 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-11-13T11:40:41.752 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:41.762 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-11-13T11:40:41.776 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-11-13T11:40:41.863 INFO:teuthology.orchestra.console.smithi195:Power off completed 2023-11-13T11:40:41.964 INFO:teuthology.orchestra.console.smithi195:Power on 2023-11-13T11:40:41.965 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-13T11:40:41.995 DEBUG:teuthology.orchestra.console.smithi195:power on output: Chassis Power Control: Up/On 2023-11-13T11:40:41.995 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:42.018 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-11-13T11:40:45.778 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:45.888 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-11-13T11:40:46.019 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:46.129 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-11-13T11:40:49.890 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:50.002 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-13T11:40:50.103 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-11-13T11:40:50.131 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:50.205 INFO:teuthology.orchestra.console.smithi181:Power on 2023-11-13T11:40:50.206 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-13T11:40:50.231 DEBUG:teuthology.orchestra.console.smithi181:power on output: Chassis Power Control: Up/On 2023-11-13T11:40:50.232 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:50.244 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2023-11-13T11:40:50.256 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-13T11:40:50.346 INFO:teuthology.orchestra.console.smithi195:Power on completed 2023-11-13T11:40:50.447 INFO:teuthology.provision.fog.smithi195:Waiting for deploy to finish 2023-11-13T11:40:50.447 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:40:54.258 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:54.370 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-13T11:40:58.374 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:40:58.534 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-11-13T11:40:58.635 INFO:teuthology.orchestra.console.smithi181:Power on completed 2023-11-13T11:40:58.736 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish 2023-11-13T11:40:58.737 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:41:05.558 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:41:13.826 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:41:20.663 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:41:28.939 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:41:35.770 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:41:44.034 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:41:50.858 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:41:59.130 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:42:05.974 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:42:14.222 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:42:21.074 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:42:29.314 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:42:36.170 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:42:44.402 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:42:51.274 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:42:59.502 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:43:06.374 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:43:14.586 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:43:21.474 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:43:29.666 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:43:36.562 DEBUG:teuthology.provision.fog.smithi195:FOG request: GET /task/active None 2023-11-13T11:43:36.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:43:39.710 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-11-13T11:43:44.726 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-11-13T11:43:44.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:43:45.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:43:47.866 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-13T11:43:48.794 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-11-13T11:43:53.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:43:54.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:43:56.922 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-13T11:43:57.854 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-11-13T11:44:02.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:03.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:05.982 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-13T11:44:06.938 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-11-13T11:44:11.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:12.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:15.066 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-13T11:44:15.998 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-11-13T11:44:21.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:22.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:24.122 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-13T11:44:25.086 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-11-13T11:44:30.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:31.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:33.182 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-13T11:44:39.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:44:57.722 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-13T11:45:03.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:45:06.778 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-13T11:45:12.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:45:15.029 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-13T11:45:15.031 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-13T11:45:31.091 WARNING:teuthology.provision.fog.smithi195:timed out 2023-11-13T11:45:37.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:45:37.317 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-13T11:45:37.318 DEBUG:teuthology.orchestra.run.smithi195:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-13T11:45:37.578 INFO:teuthology.provision.fog.smithi195:Node is ready 2023-11-13T11:45:37.578 DEBUG:teuthology.orchestra.run.smithi195:> hostname 2023-11-13T11:45:37.598 INFO:teuthology.orchestra.run.smithi195.stdout:smithi195.front.sepia.ceph.com 2023-11-13T11:45:37.599 DEBUG:teuthology.orchestra.run.smithi195:> grep smithi195.front.sepia.ceph.com /etc/hosts 2023-11-13T11:45:37.659 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.15.195 smithi195.front.sepia.ceph.com smithi195 2023-11-13T11:45:37.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195 2023-11-13T11:45:37.753 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hosts 2023-11-13T11:45:37.783 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/172.21.15.195/172.21.15.195/g' /etc/hosts 2023-11-13T11:45:37.852 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hostname 2023-11-13T11:45:37.920 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195 2023-11-13T11:45:37.986 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2023-11-13T11:45:38.044 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2023-11-13T11:45:38.045 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="8" 2023-11-13T11:45:38.045 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2023-11-13T11:45:38.045 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2023-11-13T11:45:38.045 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="8" 2023-11-13T11:45:38.046 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el8" 2023-11-13T11:45:38.046 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-13T11:45:38.046 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2023-11-13T11:45:38.046 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-13T11:45:38.046 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2023-11-13T11:45:38.047 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T11:45:38.047 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-13T11:45:38.047 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T11:45:38.048 INFO:teuthology.provision.fog.smithi195:Deploy complete! 2023-11-13T11:45:38.049 DEBUG:teuthology.parallel:result is None 2023-11-13T11:45:40.693 INFO:teuthology.provision.fog.smithi181:Node is ready 2023-11-13T11:45:40.693 DEBUG:teuthology.orchestra.run.smithi181:> hostname 2023-11-13T11:45:40.718 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com 2023-11-13T11:45:40.718 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts 2023-11-13T11:45:40.779 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181 2023-11-13T11:45:40.780 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-11-13T11:45:40.904 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts 2023-11-13T11:45:40.936 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts 2023-11-13T11:45:41.007 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname 2023-11-13T11:45:41.077 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-11-13T11:45:41.146 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-11-13T11:45:41.206 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-11-13T11:45:41.207 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2023-11-13T11:45:41.207 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-11-13T11:45:41.207 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-11-13T11:45:41.207 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2023-11-13T11:45:41.207 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-11-13T11:45:41.208 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-13T11:45:41.208 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-11-13T11:45:41.208 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-13T11:45:41.208 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-11-13T11:45:41.208 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T11:45:41.209 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-13T11:45:41.209 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T11:45:41.209 INFO:teuthology.provision.fog.smithi181:Deploy complete! 2023-11-13T11:45:41.210 DEBUG:teuthology.parallel:result is None 2023-11-13T11:45:41.364 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com 2023-11-13T11:45:41.364 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-13T11:45:41.394 INFO:teuthology.lock.ops:Checking smithi195.front.sepia.ceph.com 2023-11-13T11:45:41.394 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-13T11:45:41.420 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server 2023-11-13T11:45:41.420 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-13T11:45:41.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:45:41.486 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-11-13T11:45:41.558 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-13T11:45:41.558 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-11-13T11:45:41.617 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-11-13T11:45:41.618 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-11-13T11:45:41.677 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-11-13T11:45:41.677 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2023-11-13T11:45:41.677 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-11-13T11:45:41.677 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-11-13T11:45:41.678 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2023-11-13T11:45:41.678 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-11-13T11:45:41.678 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-13T11:45:41.678 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-11-13T11:45:41.679 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-13T11:45:41.679 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-11-13T11:45:41.679 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T11:45:41.679 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-13T11:45:41.679 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T11:45:41.680 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-11-13T11:45:41.702 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: set os type and version on server 2023-11-13T11:45:41.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-13T11:45:41.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T11:45:41.772 DEBUG:teuthology.orchestra.run.smithi195:> true 2023-11-13T11:45:41.848 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-13T11:45:41.849 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2023-11-13T11:45:41.910 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2023-11-13T11:45:41.910 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2023-11-13T11:45:41.971 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2023-11-13T11:45:41.971 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="8" 2023-11-13T11:45:41.971 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2023-11-13T11:45:41.972 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2023-11-13T11:45:41.972 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="8" 2023-11-13T11:45:41.972 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el8" 2023-11-13T11:45:41.972 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-13T11:45:41.973 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2023-11-13T11:45:41.973 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-13T11:45:41.973 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2023-11-13T11:45:41.973 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T11:45:41.973 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-13T11:45:41.974 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T11:45:41.974 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2023-11-13T11:45:42.002 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:45:42.114 INFO:teuthology.dispatcher.supervisor:Running job 7456307 2023-11-13T11:45:42.114 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner mchangir --archive /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456307 --name mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456307/orig.config.yaml 2023-11-13T11:45:42.118 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456307 2023-11-13T11:45:42.119 INFO:teuthology.dispatcher.supervisor:Job PID: 376216 2023-11-13T11:45:42.119 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-13T11:47:42.218 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:49:42.387 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:51:42.555 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:53:42.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:55:42.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:57:43.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:59:43.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T12:01:43.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T12:03:43.549 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T12:05:43.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T12:07:43.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T12:09:44.071 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T12:11:44.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T12:11:44.300 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-13T12:11:44.509 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-13T12:11:44.510 INFO:teuthology.nuke:Checking targets against current locks 2023-11-13T12:11:44.564 DEBUG:teuthology.nuke:shortname: smithi181 2023-11-13T12:11:44.564 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T12:11:44.589 DEBUG:teuthology.nuke:shortname: smithi195 2023-11-13T12:11:44.589 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T12:11:44.593 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456307', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 11:40:40.163323', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPk6iVhWybCgRAvl+o00wnOL/q1UWjhUjBLzeIVHMiD9dSAlajSSim7309Bh0StxsAcVFXW6zNM8fKtPIwea3Ts='} 2023-11-13T12:11:44.612 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456307', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 11:40:40.165220', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDF5dtBaPcLNf9wPt5NdNaQo3qQ8EQY8bomsWx7qW7Z1D3zOVujHLiasr3E19I0VtGoUqdTHYj9tRN7EBPpr95g='} 2023-11-13T12:11:44.643 INFO:teuthology.orchestra.console.smithi181:Power off 2023-11-13T12:11:44.644 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T12:11:44.668 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-11-13T12:11:44.669 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T12:11:44.685 INFO:teuthology.orchestra.console.smithi195:Power off 2023-11-13T12:11:44.686 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T12:11:44.696 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-13T12:11:44.716 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2023-11-13T12:11:44.716 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T12:11:44.738 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-11-13T12:11:44.797 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-11-13T12:11:44.840 INFO:teuthology.orchestra.console.smithi195:Power off completed 2023-11-13T12:11:44.947 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com 2023-11-13T12:11:44.989 INFO:teuthology.lock.ops:unlocked: smithi195.front.sepia.ceph.com