2023-11-15T23:59:08.865 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-15T23:59:08.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T23:59:09.004 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1631180670>, signals=[15]) 2023-11-15T23:59:09.005 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging 2023-11-15T23:59:09.005 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server 2023-11-15T23:59:09.005 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2023-11-15T23:59:09.027 INFO:teuthology.lock.ops:Start node 'smithi141.front.sepia.ceph.com' reimaging 2023-11-15T23:59:09.027 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: reset os type and version on server 2023-11-15T23:59:09.027 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2023-11-15T23:59:09.029 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /host {"name": "smithi023"} 2023-11-15T23:59:09.052 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /host {"name": "smithi141"} 2023-11-15T23:59:09.102 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-15T23:59:09.125 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-15T23:59:09.141 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_centos_8.stream (ID 3) 2023-11-15T23:59:09.142 DEBUG:teuthology.provision.fog.smithi023:FOG request: PUT /host/129 {"imageID": 3} 2023-11-15T23:59:09.165 DEBUG:teuthology.provision.fog.smithi141:Requesting image smithi_centos_8.stream (ID 3) 2023-11-15T23:59:09.165 DEBUG:teuthology.provision.fog.smithi141:FOG request: PUT /host/246 {"imageID": 3} 2023-11-15T23:59:09.193 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 8.stream 2023-11-15T23:59:09.193 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-15T23:59:09.212 INFO:teuthology.provision.fog.smithi141:Scheduling deploy of centos 8.stream 2023-11-15T23:59:09.212 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-15T23:59:09.273 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /tasktype {"name": "deploy"} 2023-11-15T23:59:09.290 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /tasktype {"name": "deploy"} 2023-11-15T23:59:09.308 DEBUG:teuthology.provision.fog.smithi023:FOG request: POST /host/129/task {"taskTypeID": 1} 2023-11-15T23:59:09.326 DEBUG:teuthology.provision.fog.smithi141:FOG request: POST /host/246/task {"taskTypeID": 1} 2023-11-15T23:59:09.367 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-15T23:59:09.372 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-15T23:59:09.479 INFO:teuthology.orchestra.console.smithi141:Power off 2023-11-15T23:59:09.480 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T23:59:09.493 INFO:teuthology.orchestra.console.smithi023:Power off 2023-11-15T23:59:09.493 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T23:59:09.501 DEBUG:teuthology.orchestra.console.smithi141:power off output: Chassis Power Control: Down/Off 2023-11-15T23:59:09.501 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:09.514 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off 2023-11-15T23:59:09.514 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:09.523 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on 2023-11-15T23:59:09.535 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2023-11-15T23:59:13.526 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:13.536 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:13.635 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on 2023-11-15T23:59:13.644 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2023-11-15T23:59:17.638 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:17.644 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:17.747 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-11-15T23:59:17.755 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2023-11-15T23:59:17.848 INFO:teuthology.orchestra.console.smithi141:Power off completed 2023-11-15T23:59:17.949 INFO:teuthology.orchestra.console.smithi141:Power on 2023-11-15T23:59:17.949 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-15T23:59:17.970 DEBUG:teuthology.orchestra.console.smithi141:power on output: Chassis Power Control: Up/On 2023-11-15T23:59:17.970 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:17.992 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-11-15T23:59:21.758 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:21.866 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2023-11-15T23:59:21.967 INFO:teuthology.orchestra.console.smithi023:Power off completed 2023-11-15T23:59:21.993 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:22.068 INFO:teuthology.orchestra.console.smithi023:Power on 2023-11-15T23:59:22.069 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-15T23:59:22.090 DEBUG:teuthology.orchestra.console.smithi023:power on output: Chassis Power Control: Up/On 2023-11-15T23:59:22.090 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:22.102 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-11-15T23:59:22.112 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2023-11-15T23:59:26.106 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:26.114 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T23:59:26.223 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2023-11-15T23:59:26.323 INFO:teuthology.orchestra.console.smithi023:Power on completed 2023-11-15T23:59:26.325 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on 2023-11-15T23:59:26.424 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish 2023-11-15T23:59:26.424 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-15T23:59:26.428 INFO:teuthology.orchestra.console.smithi141:Power on completed 2023-11-15T23:59:26.529 INFO:teuthology.provision.fog.smithi141:Waiting for deploy to finish 2023-11-15T23:59:26.529 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-15T23:59:41.526 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-15T23:59:41.612 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-15T23:59:56.632 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-15T23:59:56.691 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:00:11.750 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:00:11.775 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:00:26.859 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:00:26.862 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:00:41.998 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:00:42.000 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:00:57.130 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:00:57.131 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:01:12.270 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:01:12.272 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:01:27.394 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:01:35.107 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:01:50.262 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:01:50.265 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:02:05.402 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:02:05.405 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-11-16T00:02:05.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:02:08.602 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-11-16T00:02:14.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:02:17.659 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-11-16T00:02:20.533 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:02:23.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:02:26.714 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-11-16T00:02:32.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:02:35.626 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None 2023-11-16T00:02:35.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:02:35.770 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-11-16T00:02:35.846 ERROR:teuthology.orchestra.connection:Error authenticating with smithi023.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-16T00:02:41.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:02:44.830 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-11-16T00:02:50.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:02:53.915 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-11-16T00:02:59.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:03:18.266 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-11-16T00:03:24.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:03:27.322 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-11-16T00:03:33.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:03:34.907 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-16T00:03:34.907 DEBUG:teuthology.orchestra.run.smithi141:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-16T00:03:39.850 WARNING:teuthology.provision.fog.smithi023:timed out 2023-11-16T00:03:45.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:03:48.922 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2023-11-16T00:03:54.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:03:57.978 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2023-11-16T00:03:59.770 INFO:teuthology.provision.fog.smithi141:Node is ready 2023-11-16T00:04:10.761 DEBUG:teuthology.orchestra.run.smithi141:> hostname 2023-11-16T00:04:10.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:04:10.794 INFO:teuthology.orchestra.run.smithi141.stdout:smithi141.front.sepia.ceph.com 2023-11-16T00:04:10.799 DEBUG:teuthology.orchestra.run.smithi141:> grep smithi141.front.sepia.ceph.com /etc/hosts 2023-11-16T00:04:10.856 INFO:teuthology.orchestra.run.smithi141.stdout:172.21.15.141 smithi141.front.sepia.ceph.com smithi141 2023-11-16T00:04:10.856 DEBUG:teuthology.orchestra.run.smithi141:> sudo hostname smithi141 2023-11-16T00:04:10.971 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/smithi141.front.sepia.ceph.com/smithi141/g' /etc/hosts 2023-11-16T00:04:11.002 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/172.21.15.141/172.21.15.141/g' /etc/hosts 2023-11-16T00:04:11.072 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/smithi141.front.sepia.ceph.com/smithi141/g' /etc/hostname 2023-11-16T00:04:11.140 DEBUG:teuthology.orchestra.run.smithi141:> sudo hostname smithi141 2023-11-16T00:04:11.208 DEBUG:teuthology.orchestra.run.smithi141:> cat /etc/os-release 2023-11-16T00:04:11.266 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="CentOS Stream" 2023-11-16T00:04:11.266 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8" 2023-11-16T00:04:11.266 INFO:teuthology.orchestra.run.smithi141.stdout:ID="centos" 2023-11-16T00:04:11.266 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="rhel fedora" 2023-11-16T00:04:11.266 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8" 2023-11-16T00:04:11.266 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8" 2023-11-16T00:04:11.267 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-16T00:04:11.267 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31" 2023-11-16T00:04:11.267 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-16T00:04:11.267 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://centos.org/" 2023-11-16T00:04:11.267 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T00:04:11.267 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T00:04:11.267 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-16T00:04:11.268 INFO:teuthology.provision.fog.smithi141:Deploy complete! 2023-11-16T00:04:11.269 DEBUG:teuthology.parallel:result is None 2023-11-16T00:05:10.771 WARNING:teuthology.provision.fog.smithi023:timed out 2023-11-16T00:05:16.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:05:17.015 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-16T00:05:17.016 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-16T00:05:17.298 INFO:teuthology.provision.fog.smithi023:Node is ready 2023-11-16T00:05:17.298 DEBUG:teuthology.orchestra.run.smithi023:> hostname 2023-11-16T00:05:17.318 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com 2023-11-16T00:05:17.318 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts 2023-11-16T00:05:17.378 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023 2023-11-16T00:05:17.378 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023 2023-11-16T00:05:17.488 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts 2023-11-16T00:05:17.518 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts 2023-11-16T00:05:17.586 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname 2023-11-16T00:05:17.656 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023 2023-11-16T00:05:17.723 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2023-11-16T00:05:17.781 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2023-11-16T00:05:17.782 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="8" 2023-11-16T00:05:17.782 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2023-11-16T00:05:17.782 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2023-11-16T00:05:17.782 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="8" 2023-11-16T00:05:17.783 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el8" 2023-11-16T00:05:17.783 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-16T00:05:17.783 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2023-11-16T00:05:17.784 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-16T00:05:17.785 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2023-11-16T00:05:17.785 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T00:05:17.785 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T00:05:17.786 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-16T00:05:17.786 INFO:teuthology.provision.fog.smithi023:Deploy complete! 2023-11-16T00:05:17.787 DEBUG:teuthology.parallel:result is None 2023-11-16T00:05:17.975 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com 2023-11-16T00:05:17.975 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-16T00:05:18.002 INFO:teuthology.lock.ops:Checking smithi141.front.sepia.ceph.com 2023-11-16T00:05:18.002 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-16T00:05:18.019 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server 2023-11-16T00:05:18.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2023-11-16T00:05:18.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:05:18.089 DEBUG:teuthology.orchestra.run.smithi023:> true 2023-11-16T00:05:18.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com' 2023-11-16T00:05:18.156 DEBUG:teuthology.orchestra.run.smithi023:> uname -m 2023-11-16T00:05:18.214 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2023-11-16T00:05:18.214 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2023-11-16T00:05:18.272 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="8" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="8" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el8" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T00:05:18.273 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T00:05:18.274 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-16T00:05:18.274 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2023-11-16T00:05:18.310 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: set os type and version on server 2023-11-16T00:05:18.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-11-16T00:05:18.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T00:05:18.377 DEBUG:teuthology.orchestra.run.smithi141:> true 2023-11-16T00:05:18.509 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-11-16T00:05:18.510 DEBUG:teuthology.orchestra.run.smithi141:> uname -m 2023-11-16T00:05:18.568 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64 2023-11-16T00:05:18.568 DEBUG:teuthology.orchestra.run.smithi141:> cat /etc/os-release 2023-11-16T00:05:18.627 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="CentOS Stream" 2023-11-16T00:05:18.627 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:ID="centos" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="rhel fedora" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://centos.org/" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T00:05:18.628 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-16T00:05:18.629 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2023-11-16T00:05:18.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:05:18.739 INFO:teuthology.dispatcher.supervisor:Running job 7459545 2023-11-16T00:05:18.739 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459545 --name yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459545/orig.config.yaml 2023-11-16T00:05:18.743 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459545 2023-11-16T00:05:18.744 INFO:teuthology.dispatcher.supervisor:Job PID: 843838 2023-11-16T00:05:18.744 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-16T00:07:18.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:09:18.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:11:18.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:13:19.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:15:19.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:17:19.157 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:19:19.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:21:19.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:23:19.374 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:25:19.450 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:27:19.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:29:19.600 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:31:19.669 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:33:19.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:35:19.809 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:37:19.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:39:19.957 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:41:20.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:43:20.104 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:45:20.169 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:47:20.240 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:49:20.320 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:51:20.402 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:53:20.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:55:20.570 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:57:20.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T00:59:20.721 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T01:01:20.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T01:03:20.854 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T01:05:20.920 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T01:05:20.988 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-11-16T01:05:21.171 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-16T01:05:21.171 INFO:teuthology.nuke:Checking targets against current locks 2023-11-16T01:05:21.224 DEBUG:teuthology.nuke:shortname: smithi023 2023-11-16T01:05:21.224 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T01:05:21.247 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi023.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459545', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 23:59:08.064610', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHUPbL5cwI/bi3MAIT/6vf1SKwhGMvmWAKIAD1PwoLWuxxBtuc9FXhUE+AnSsm9qJilZibo3N/sF/qJvDC4dKdQ='} 2023-11-16T01:05:21.251 DEBUG:teuthology.nuke:shortname: smithi141 2023-11-16T01:05:21.251 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T01:05:21.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi141.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459545', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 23:59:08.062654', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEHa7uWr+zDlrYC0zPvQXoMEAJUz9KijHQraaYI7UufXm9oZg76n+x7nMjlP1K0paw4HGLk5kta7TIcQAxlv04Y='} 2023-11-16T01:05:21.300 INFO:teuthology.orchestra.console.smithi023:Power off 2023-11-16T01:05:21.300 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-16T01:05:21.326 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off 2023-11-16T01:05:21.326 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T01:05:21.345 INFO:teuthology.orchestra.console.smithi141:Power off 2023-11-16T01:05:21.345 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-16T01:05:21.358 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2023-11-16T01:05:21.370 DEBUG:teuthology.orchestra.console.smithi141:power off output: Chassis Power Control: Down/Off 2023-11-16T01:05:21.370 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T01:05:21.392 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-11-16T01:05:21.459 INFO:teuthology.orchestra.console.smithi023:Power off completed 2023-11-16T01:05:21.493 INFO:teuthology.orchestra.console.smithi141:Power off completed 2023-11-16T01:05:21.603 INFO:teuthology.lock.ops:unlocked: smithi023.front.sepia.ceph.com 2023-11-16T01:05:21.637 INFO:teuthology.lock.ops:unlocked: smithi141.front.sepia.ceph.com