2023-08-15T03:58:02.054 INFO:root:teuthology version: 0.0.1.dev183+g5435569 2023-08-15T03:58:02.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T03:58:02.229 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f37eb860f70>, signals=[15]) 2023-08-15T03:58:02.230 INFO:teuthology.lock.ops:Start node 'smithi079.front.sepia.ceph.com' reimaging 2023-08-15T03:58:02.230 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: reset os type and version on server 2023-08-15T03:58:02.231 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2023-08-15T03:58:02.259 INFO:teuthology.lock.ops:Start node 'smithi170.front.sepia.ceph.com' reimaging 2023-08-15T03:58:02.260 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: reset os type and version on server 2023-08-15T03:58:02.260 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-08-15T03:58:02.261 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /host {"name": "smithi079"} 2023-08-15T03:58:02.284 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /host {"name": "smithi170"} 2023-08-15T03:58:02.349 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-15T03:58:02.358 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-15T03:58:02.404 DEBUG:teuthology.provision.fog.smithi079:Requesting image smithi_centos_8.stream (ID 3) 2023-08-15T03:58:02.404 DEBUG:teuthology.provision.fog.smithi079:FOG request: PUT /host/184 {"imageID": 3} 2023-08-15T03:58:02.407 DEBUG:teuthology.provision.fog.smithi170:Requesting image smithi_centos_8.stream (ID 3) 2023-08-15T03:58:02.407 DEBUG:teuthology.provision.fog.smithi170:FOG request: PUT /host/275 {"imageID": 3} 2023-08-15T03:58:02.451 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of centos 8.stream 2023-08-15T03:58:02.451 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:58:02.455 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 8.stream 2023-08-15T03:58:02.456 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:58:02.519 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /tasktype {"name": "deploy"} 2023-08-15T03:58:02.522 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /tasktype {"name": "deploy"} 2023-08-15T03:58:02.562 DEBUG:teuthology.provision.fog.smithi170:FOG request: POST /host/275/task {"taskTypeID": 1} 2023-08-15T03:58:02.566 DEBUG:teuthology.provision.fog.smithi079:FOG request: POST /host/184/task {"taskTypeID": 1} 2023-08-15T03:58:02.618 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:58:02.622 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:58:02.738 INFO:teuthology.orchestra.console.smithi079:Power off 2023-08-15T03:58:02.739 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T03:58:02.757 INFO:teuthology.orchestra.console.smithi170:Power off 2023-08-15T03:58:02.758 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T03:58:02.770 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2023-08-15T03:58:02.771 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T03:58:02.787 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2023-08-15T03:58:02.788 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T03:58:02.799 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-08-15T03:58:02.810 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2023-08-15T03:58:02.901 INFO:teuthology.orchestra.console.smithi079:Power off completed 2023-08-15T03:58:02.911 INFO:teuthology.orchestra.console.smithi170:Power off completed 2023-08-15T03:58:03.003 INFO:teuthology.orchestra.console.smithi079:Power on 2023-08-15T03:58:03.003 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-15T03:58:03.012 INFO:teuthology.orchestra.console.smithi170:Power on 2023-08-15T03:58:03.013 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-15T03:58:03.034 DEBUG:teuthology.orchestra.console.smithi170:power on output: Chassis Power Control: Up/On 2023-08-15T03:58:03.034 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T03:58:03.042 DEBUG:teuthology.orchestra.console.smithi079:power on output: Chassis Power Control: Up/On 2023-08-15T03:58:03.043 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T03:58:03.063 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-08-15T03:58:03.090 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2023-08-15T03:58:07.068 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T03:58:07.092 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T03:58:07.178 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-08-15T03:58:07.202 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-08-15T03:58:07.303 INFO:teuthology.orchestra.console.smithi170:Power on completed 2023-08-15T03:58:07.404 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish 2023-08-15T03:58:07.404 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:58:11.183 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T03:58:11.314 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-08-15T03:58:11.415 INFO:teuthology.orchestra.console.smithi079:Power on completed 2023-08-15T03:58:11.516 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish 2023-08-15T03:58:11.516 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:58:22.475 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:58:26.576 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:58:37.544 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:58:41.644 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:58:52.612 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:58:56.700 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:59:07.680 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:59:11.760 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:59:22.748 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:59:26.820 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:59:37.816 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:59:41.891 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T03:59:52.889 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T03:59:56.964 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T04:00:07.964 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T04:00:12.052 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-15T04:00:12.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:00:15.184 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:00:21.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:00:23.036 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T04:00:24.240 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:00:30.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:00:33.296 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:00:38.120 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T04:00:39.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:00:42.356 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:00:48.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:00:51.440 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:00:53.198 DEBUG:teuthology.provision.fog.smithi170:FOG request: GET /task/active None 2023-08-15T04:00:53.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:00:56.336 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-08-15T04:00:57.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:00.496 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:01:02.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:05.392 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-08-15T04:01:06.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:09.552 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:01:11.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:14.448 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-08-15T04:01:15.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:18.612 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:01:20.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:23.504 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-08-15T04:01:24.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:29.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:30.704 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:01:32.560 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-08-15T04:01:36.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:38.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:39.760 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:01:45.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:48.849 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-15T04:01:54.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:01:54.966 ERROR:teuthology.orchestra.connection:Error authenticating with smithi079.front.sepia.ceph.com: Authentication failed. 2023-08-15T04:01:54.967 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-15T04:01:54.968 DEBUG:teuthology.orchestra.run.smithi079:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-15T04:01:54.969 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:01:55.977 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:01:56.982 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:01:57.991 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:01:58.999 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:00.010 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:01.020 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:02.032 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:03.039 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:04.046 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:05.054 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:06.061 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:07.071 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:08.078 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:09.084 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:10.091 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:11.100 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:12.108 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:13.117 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:14.129 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:15.137 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:16.152 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:17.165 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:18.171 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:19.182 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:20.203 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:21.220 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:22.244 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:23.252 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:24.261 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:25.268 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:26.275 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:27.283 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:28.291 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:29.300 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:30.308 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:31.316 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:32.322 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:33.329 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:34.335 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:35.343 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:36.349 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:37.360 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:38.372 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:38.566 WARNING:teuthology.provision.fog.smithi170:timed out 2023-08-15T04:02:39.383 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:40.391 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:41.400 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:42.409 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:43.419 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:44.429 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:44.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:02:44.804 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-15T04:02:44.804 DEBUG:teuthology.orchestra.run.smithi170:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-15T04:02:45.077 INFO:teuthology.provision.fog.smithi170:Node is ready 2023-08-15T04:02:45.077 DEBUG:teuthology.orchestra.run.smithi170:> hostname 2023-08-15T04:02:45.096 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com 2023-08-15T04:02:45.098 DEBUG:teuthology.orchestra.run.smithi170:> grep smithi170.front.sepia.ceph.com /etc/hosts 2023-08-15T04:02:45.156 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170 2023-08-15T04:02:45.157 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname smithi170 2023-08-15T04:02:45.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hosts 2023-08-15T04:02:45.280 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/172.21.15.170/172.21.15.170/g' /etc/hosts 2023-08-15T04:02:45.353 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hostname 2023-08-15T04:02:45.422 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname smithi170 2023-08-15T04:02:45.436 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:45.490 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2023-08-15T04:02:45.549 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2023-08-15T04:02:45.550 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8" 2023-08-15T04:02:45.550 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2023-08-15T04:02:45.550 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2023-08-15T04:02:45.550 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8" 2023-08-15T04:02:45.550 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2023-08-15T04:02:45.550 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T04:02:45.551 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2023-08-15T04:02:45.551 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T04:02:45.551 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2023-08-15T04:02:45.551 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T04:02:45.551 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T04:02:45.551 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T04:02:45.552 INFO:teuthology.provision.fog.smithi170:Deploy complete! 2023-08-15T04:02:45.553 DEBUG:teuthology.parallel:result is None 2023-08-15T04:02:46.444 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:47.451 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:48.461 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:02:49.471 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T04:05:04.756 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-08-15T04:05:04.757 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi079 2023-08-15T04:05:13.764 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-08-15T04:05:13.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T04:05:13.830 DEBUG:teuthology.orchestra.run.smithi079:> true 2023-08-15T04:05:13.894 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-08-15T04:05:13.894 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-15T04:05:13.894 DEBUG:teuthology.orchestra.run.smithi079:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-15T04:05:13.947 INFO:teuthology.provision.fog.smithi079:Node is ready 2023-08-15T04:05:13.948 DEBUG:teuthology.orchestra.run.smithi079:> hostname 2023-08-15T04:05:14.002 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079 2023-08-15T04:05:14.002 DEBUG:teuthology.orchestra.run.smithi079:> grep smithi079 /etc/hosts 2023-08-15T04:05:14.057 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079 smithi079 2023-08-15T04:05:14.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2023-08-15T04:05:14.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079/smithi079/g' /etc/hosts 2023-08-15T04:05:14.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/172.21.15.79/172.21.15.79/g' /etc/hosts 2023-08-15T04:05:14.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079/smithi079/g' /etc/hostname 2023-08-15T04:05:14.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2023-08-15T04:05:14.377 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2023-08-15T04:05:14.433 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Red Hat Enterprise Linux" 2023-08-15T04:05:14.433 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8.4 (Ootpa)" 2023-08-15T04:05:14.434 INFO:teuthology.orchestra.run.smithi079.stdout:ID="rhel" 2023-08-15T04:05:14.434 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="fedora" 2023-08-15T04:05:14.434 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8.4" 2023-08-15T04:05:14.434 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2023-08-15T04:05:14.435 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-08-15T04:05:14.436 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2023-08-15T04:05:14.436 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-08-15T04:05:14.436 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.redhat.com/" 2023-08-15T04:05:14.436 INFO:teuthology.orchestra.run.smithi079.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-15T04:05:14.436 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T04:05:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-08-15T04:05:14.437 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T04:05:14.437 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-08-15T04:05:14.437 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-15T04:05:14.437 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-08-15T04:05:14.438 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi079's OS to be centos 8 but found rhel 8.4 2023-08-15T04:05:14.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T04:05:14.578 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5cc487466c5d4e0b930d65516cf3e607 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi079's OS to be centos 8 but found rhel 8.4 2023-08-15T04:05:14.580 INFO:teuthology.nuke:Checking targets against current locks 2023-08-15T04:05:14.649 DEBUG:teuthology.nuke:shortname: smithi079 2023-08-15T04:05:14.650 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-15T04:05:14.673 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-15_01:22:18-fs-wip-yuri8-testing-2023-08-11-0834-pacific-distro-default-smithi/7367793', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-15 04:00:03.327062', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRgmRhutVphkjR4EmKBHaOD1zRDyhrxAvUfcshxiM791RF9QUPX2BcYxNSI+ucS/DMtt6kSgipOOX54ut0tyEoVG6fIx/nGkkleEXXjJ5x9cmv80LTXvFXeC8Nva1yXtHElFwSNr6PVBYjcMO02Q0nMhaSMIb7fdecj587H9JpTUI1V3Hjj4KGNYkd5TlMzcvi+zWFyjRLCzufmppbbwM+rT9Gok++WuXIE8sgO2WvAa8eNnK17C7qg9UNa7Nh41ipRJphfdNwib21tLg1V7EnayCObnQ7GfDZw1FZEkpLeNOS7aTzFrf41ijiepEXKZ+yqBHWKTbgh3/KZwZowuyP6XhLDQqTHH6NBFyKuRtx9ZlEKKb/OQPWQS7XnTLwqadIcKiwYU8BQVpRUsfrNQFKHok/YPkFvb3O8iNosIVJXcsI4mmiYEaop3evZA2i2zV3F4ETmJ5eEFjJ6ijZ1H+1ZqOZzv2ZrpyN0ZHFOw1NHOUn3TyG15mH+I3ETFfNSKs='} 2023-08-15T04:05:14.678 DEBUG:teuthology.nuke:shortname: smithi170 2023-08-15T04:05:14.678 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-15T04:05:14.704 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-15_01:22:18-fs-wip-yuri8-testing-2023-08-11-0834-pacific-distro-default-smithi/7367788', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-15 03:58:01.205135', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzqSSG+/YOxfk8VJXK7n77JiGDkAGKQfcnFpf51xyEmPxgjE6vzGuL9L9B5W/oPRoqok5kD187YlRhcMB39nq08zYqf/MCNP0iTOrNfugycgsXbNoi0xOKCmbiDi1SZcWagkH4WvDzjE0eiVJa0B0s6dsbNoWD8c2e6Md+lr/59ISmAXRGzpoaKHbE8EO1uVPFK+zJ0HkOiwtU0mH6cQA1gUc8KYYdgeLDhXxXmssZ/HNLEGeT8neFZkFy+b8Ev3cBGWVZ48SHOcPLiyJ+XD0XSkBuQzkJYOS1Lz/ZevsHol3qwXu5aKUmADQHTLE0wTDLcDjNLms+SfVxfWj3mX+HzYVXptgHO/ewXQhAxzOKy0yHC29MTwt1XnYAZKrNTBMBrasjVieAtizgRklt3Pb/CvTMYzvuTp69YPcH1ka+RZjbXtKdb30VyxRXbRQJOxoaWHI4OxiLJL9A8W7kSCwkuFkLAKPjYs1uQvkuc1SHklMhssuZ11LcOT8bIaVrY88='} 2023-08-15T04:05:14.731 INFO:teuthology.orchestra.console.smithi079:Power off 2023-08-15T04:05:14.732 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T04:05:14.758 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2023-08-15T04:05:14.759 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T04:05:14.779 INFO:teuthology.orchestra.console.smithi170:Power off 2023-08-15T04:05:14.781 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T04:05:14.794 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-08-15T04:05:14.810 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2023-08-15T04:05:14.810 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T04:05:14.838 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-08-15T04:05:18.796 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T04:05:18.841 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T04:05:18.908 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-08-15T04:05:18.953 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-08-15T04:05:22.912 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T04:05:22.955 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T04:05:23.030 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-08-15T04:05:23.068 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2023-08-15T04:05:23.169 INFO:teuthology.orchestra.console.smithi170:Power off completed 2023-08-15T04:05:23.320 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com 2023-08-15T04:05:27.032 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T04:05:27.145 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-08-15T04:05:27.246 INFO:teuthology.orchestra.console.smithi079:Power off completed 2023-08-15T04:05:27.403 INFO:teuthology.lock.ops:unlocked: smithi079.front.sepia.ceph.com 2023-08-15T04:05:27.674 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi079's OS to be centos 8 but found rhel 8.4