2023-08-07T19:26:53.178 INFO:root:teuthology version: 0.0.1.dev179+gce5f6d5 2023-08-07T19:26:53.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:26:53.358 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb4e6259280>, signals=[15]) 2023-08-07T19:26:53.359 INFO:teuthology.lock.ops:Start node 'smithi105.front.sepia.ceph.com' reimaging 2023-08-07T19:26:53.359 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: reset os type and version on server 2023-08-07T19:26:53.359 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-08-07T19:26:53.385 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging 2023-08-07T19:26:53.386 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server 2023-08-07T19:26:53.386 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-08-07T19:26:53.388 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /host {"name": "smithi105"} 2023-08-07T19:26:53.410 INFO:teuthology.lock.ops:Start node 'smithi178.front.sepia.ceph.com' reimaging 2023-08-07T19:26:53.410 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: reset os type and version on server 2023-08-07T19:26:53.411 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-08-07T19:26:53.412 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /host {"name": "smithi146"} 2023-08-07T19:26:53.436 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /host {"name": "smithi178"} 2023-08-07T19:26:53.480 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-08-07T19:26:53.491 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-08-07T19:26:53.510 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-08-07T19:26:53.530 DEBUG:teuthology.provision.fog.smithi146:Requesting image smithi_rhel_8.6 (ID 5) 2023-08-07T19:26:53.530 DEBUG:teuthology.provision.fog.smithi146:FOG request: PUT /host/251 {"imageID": 5} 2023-08-07T19:26:53.531 DEBUG:teuthology.provision.fog.smithi105:Requesting image smithi_rhel_8.6 (ID 5) 2023-08-07T19:26:53.531 DEBUG:teuthology.provision.fog.smithi105:FOG request: PUT /host/210 {"imageID": 5} 2023-08-07T19:26:53.549 DEBUG:teuthology.provision.fog.smithi178:Requesting image smithi_rhel_8.6 (ID 5) 2023-08-07T19:26:53.549 DEBUG:teuthology.provision.fog.smithi178:FOG request: PUT /host/283 {"imageID": 5} 2023-08-07T19:26:53.574 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of rhel 8.6 2023-08-07T19:26:53.574 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:26:53.578 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of rhel 8.6 2023-08-07T19:26:53.578 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:26:53.592 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of rhel 8.6 2023-08-07T19:26:53.592 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:26:53.631 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /tasktype {"name": "deploy"} 2023-08-07T19:26:53.634 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /tasktype {"name": "deploy"} 2023-08-07T19:26:53.642 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /tasktype {"name": "deploy"} 2023-08-07T19:26:53.674 DEBUG:teuthology.provision.fog.smithi146:FOG request: POST /host/251/task {"taskTypeID": 1} 2023-08-07T19:26:53.681 DEBUG:teuthology.provision.fog.smithi105:FOG request: POST /host/210/task {"taskTypeID": 1} 2023-08-07T19:26:53.683 DEBUG:teuthology.provision.fog.smithi178:FOG request: POST /host/283/task {"taskTypeID": 1} 2023-08-07T19:26:53.762 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:26:53.784 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:26:53.799 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:26:53.867 INFO:teuthology.orchestra.console.smithi105:Power off 2023-08-07T19:26:53.867 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-07T19:26:53.888 INFO:teuthology.orchestra.console.smithi146:Power off 2023-08-07T19:26:53.889 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-07T19:26:53.899 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2023-08-07T19:26:53.900 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:26:53.913 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2023-08-07T19:26:53.913 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:26:53.930 INFO:teuthology.orchestra.console.smithi178:Power off 2023-08-07T19:26:53.930 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-07T19:26:53.938 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2023-08-07T19:26:53.939 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-08-07T19:26:53.953 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2023-08-07T19:26:53.953 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:26:53.976 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-08-07T19:26:57.940 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:26:57.949 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:26:57.977 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:26:58.051 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2023-08-07T19:26:58.060 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-08-07T19:26:58.089 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-08-07T19:27:02.052 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:02.063 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:02.090 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:02.166 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2023-08-07T19:27:02.180 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-08-07T19:27:02.201 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-08-07T19:27:02.281 INFO:teuthology.orchestra.console.smithi146:Power off completed 2023-08-07T19:27:02.302 INFO:teuthology.orchestra.console.smithi178:Power off completed 2023-08-07T19:27:02.383 INFO:teuthology.orchestra.console.smithi146:Power on 2023-08-07T19:27:02.383 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-07T19:27:02.403 INFO:teuthology.orchestra.console.smithi178:Power on 2023-08-07T19:27:02.404 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-07T19:27:02.416 DEBUG:teuthology.orchestra.console.smithi146:power on output: Chassis Power Control: Up/On 2023-08-07T19:27:02.417 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:02.431 DEBUG:teuthology.orchestra.console.smithi178:power on output: Chassis Power Control: Up/On 2023-08-07T19:27:02.431 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:02.444 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-08-07T19:27:02.457 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-08-07T19:27:06.168 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:06.279 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2023-08-07T19:27:06.379 INFO:teuthology.orchestra.console.smithi105:Power off completed 2023-08-07T19:27:06.445 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:06.458 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:06.480 INFO:teuthology.orchestra.console.smithi105:Power on 2023-08-07T19:27:06.480 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-07T19:27:06.503 DEBUG:teuthology.orchestra.console.smithi105:power on output: Chassis Power Control: Up/On 2023-08-07T19:27:06.504 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:06.527 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2023-08-07T19:27:06.556 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-08-07T19:27:06.569 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-08-07T19:27:10.532 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:10.557 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:10.570 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:10.643 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2023-08-07T19:27:10.669 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-08-07T19:27:10.684 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-08-07T19:27:10.770 INFO:teuthology.orchestra.console.smithi146:Power on completed 2023-08-07T19:27:10.785 INFO:teuthology.orchestra.console.smithi178:Power on completed 2023-08-07T19:27:10.871 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish 2023-08-07T19:27:10.871 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:27:10.887 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish 2023-08-07T19:27:10.887 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:27:14.644 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T19:27:14.754 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2023-08-07T19:27:14.855 INFO:teuthology.orchestra.console.smithi105:Power on completed 2023-08-07T19:27:14.955 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish 2023-08-07T19:27:14.955 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:27:25.948 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:27:25.949 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:27:30.022 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:27:41.032 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:27:41.033 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:27:45.096 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:27:56.104 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:27:56.106 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:28:00.162 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:28:11.172 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:28:11.173 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:28:15.228 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:28:26.244 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:28:26.246 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:28:30.292 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:28:41.320 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:28:41.321 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:28:45.352 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:28:56.389 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:28:56.390 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:29:00.416 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:29:11.456 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:29:11.457 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:29:15.476 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:29:26.528 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:29:26.529 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:29:30.540 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:29:41.592 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:29:41.593 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:29:45.609 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:29:56.654 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-08-07T19:29:56.655 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-07T19:29:56.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:29:56.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:29:56.924 ERROR:teuthology.orchestra.connection:Error authenticating with smithi178.front.sepia.ceph.com: Authentication failed. 2023-08-07T19:29:56.925 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-07T19:29:56.925 DEBUG:teuthology.orchestra.run.smithi178:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-07T19:29:56.926 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-07T19:29:59.760 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-07T19:30:00.669 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-08-07T19:30:00.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:03.792 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-08-07T19:30:05.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:08.820 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-07T19:30:09.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:12.848 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-08-07T19:30:14.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:17.872 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-07T19:30:18.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:21.904 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-08-07T19:30:23.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:26.928 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-07T19:30:27.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:30.960 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-08-07T19:30:32.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:35.984 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-07T19:30:36.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:40.016 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-08-07T19:30:41.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:45.044 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-07T19:30:46.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:49.072 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-08-07T19:30:51.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:30:55.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:31:51.047 WARNING:teuthology.provision.fog.smithi146:timed out 2023-08-07T19:31:52.048 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-08-07T19:31:52.049 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi178 2023-08-07T19:31:55.078 WARNING:teuthology.provision.fog.smithi105:timed out 2023-08-07T19:31:57.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:31:57.377 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-07T19:31:57.377 DEBUG:teuthology.orchestra.run.smithi146:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-07T19:31:57.671 INFO:teuthology.provision.fog.smithi146:Node is ready 2023-08-07T19:31:57.671 DEBUG:teuthology.orchestra.run.smithi146:> hostname 2023-08-07T19:31:57.688 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com 2023-08-07T19:31:57.689 DEBUG:teuthology.orchestra.run.smithi146:> grep smithi146.front.sepia.ceph.com /etc/hosts 2023-08-07T19:31:57.747 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146 2023-08-07T19:31:57.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146 2023-08-07T19:31:57.851 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hosts 2023-08-07T19:31:57.879 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/172.21.15.146/172.21.15.146/g' /etc/hosts 2023-08-07T19:31:57.946 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hostname 2023-08-07T19:31:58.013 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146 2023-08-07T19:31:58.078 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-08-07T19:31:58.133 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Red Hat Enterprise Linux" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8.6 (Ootpa)" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:ID="rhel" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="fedora" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8.6" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.redhat.com/" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-07T19:31:58.134 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-07T19:31:58.135 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-07T19:31:58.135 INFO:teuthology.provision.fog.smithi146:Deploy complete! 2023-08-07T19:31:58.136 DEBUG:teuthology.parallel:result is None 2023-08-07T19:32:01.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-08-07T19:32:01.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:32:01.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:32:01.394 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-08-07T19:32:01.407 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-07T19:32:01.407 DEBUG:teuthology.orchestra.run.smithi105:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-07T19:32:01.663 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-08-07T19:32:01.664 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-07T19:32:01.664 DEBUG:teuthology.orchestra.run.smithi178:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-07T19:32:01.682 INFO:teuthology.provision.fog.smithi178:Node is ready 2023-08-07T19:32:01.682 DEBUG:teuthology.orchestra.run.smithi178:> hostname 2023-08-07T19:32:01.724 INFO:teuthology.provision.fog.smithi105:Node is ready 2023-08-07T19:32:01.724 DEBUG:teuthology.orchestra.run.smithi105:> hostname 2023-08-07T19:32:01.740 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com 2023-08-07T19:32:01.740 DEBUG:teuthology.orchestra.run.smithi178:> grep smithi178.front.sepia.ceph.com /etc/hosts 2023-08-07T19:32:01.743 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105.front.sepia.ceph.com 2023-08-07T19:32:01.743 DEBUG:teuthology.orchestra.run.smithi105:> grep smithi105.front.sepia.ceph.com /etc/hosts 2023-08-07T19:32:01.799 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178 2023-08-07T19:32:01.799 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname smithi178 2023-08-07T19:32:01.802 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105 smithi105.front.sepia.ceph.com smithi105 2023-08-07T19:32:01.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname smithi105 2023-08-07T19:32:01.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/smithi105.front.sepia.ceph.com/smithi105/g' /etc/hosts 2023-08-07T19:32:01.930 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/smithi178.front.sepia.ceph.com/smithi178/g' /etc/hosts 2023-08-07T19:32:01.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/172.21.15.105/172.21.15.105/g' /etc/hosts 2023-08-07T19:32:01.959 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/172.21.15.178/172.21.15.178/g' /etc/hosts 2023-08-07T19:32:02.013 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/smithi105.front.sepia.ceph.com/smithi105/g' /etc/hostname 2023-08-07T19:32:02.028 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/smithi178.front.sepia.ceph.com/smithi178/g' /etc/hostname 2023-08-07T19:32:02.081 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname smithi105 2023-08-07T19:32:02.096 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname smithi178 2023-08-07T19:32:02.147 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2023-08-07T19:32:02.163 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-08-07T19:32:02.203 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Red Hat Enterprise Linux" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8.6 (Ootpa)" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:ID="rhel" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="fedora" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8.6" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.redhat.com/" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-07T19:32:02.204 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-07T19:32:02.205 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-07T19:32:02.205 INFO:teuthology.provision.fog.smithi105:Deploy complete! 2023-08-07T19:32:02.205 DEBUG:teuthology.parallel:result is None 2023-08-07T19:32:02.221 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux" 2023-08-07T19:32:02.221 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)" 2023-08-07T19:32:02.221 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel" 2023-08-07T19:32:02.221 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora" 2023-08-07T19:32:02.221 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6" 2023-08-07T19:32:02.221 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2023-08-07T19:32:02.221 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/" 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-07T19:32:02.222 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-07T19:32:02.222 INFO:teuthology.provision.fog.smithi178:Deploy complete! 2023-08-07T19:32:02.223 DEBUG:teuthology.parallel:result is None 2023-08-07T19:32:02.387 INFO:teuthology.lock.ops:Checking smithi105.front.sepia.ceph.com 2023-08-07T19:32:02.388 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-07T19:32:02.415 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com 2023-08-07T19:32:02.415 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-07T19:32:02.431 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com 2023-08-07T19:32:02.432 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-07T19:32:02.449 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: set os type and version on server 2023-08-07T19:32:02.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-08-07T19:32:02.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:32:02.519 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-08-07T19:32:02.588 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-08-07T19:32:02.588 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2023-08-07T19:32:02.644 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2023-08-07T19:32:02.645 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2023-08-07T19:32:02.700 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Red Hat Enterprise Linux" 2023-08-07T19:32:02.700 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8.6 (Ootpa)" 2023-08-07T19:32:02.701 INFO:teuthology.orchestra.run.smithi105.stdout:ID="rhel" 2023-08-07T19:32:02.701 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="fedora" 2023-08-07T19:32:02.701 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8.6" 2023-08-07T19:32:02.701 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2023-08-07T19:32:02.701 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-07T19:32:02.701 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2023-08-07T19:32:02.701 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-07T19:32:02.701 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.redhat.com/" 2023-08-07T19:32:02.702 INFO:teuthology.orchestra.run.smithi105.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-07T19:32:02.702 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-07T19:32:02.702 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-08-07T19:32:02.702 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-07T19:32:02.702 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-07T19:32:02.702 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-07T19:32:02.702 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-07T19:32:02.703 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-08-07T19:32:02.723 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server 2023-08-07T19:32:02.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-08-07T19:32:02.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:32:02.790 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-08-07T19:32:02.860 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-08-07T19:32:02.860 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2023-08-07T19:32:02.919 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2023-08-07T19:32:02.920 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Red Hat Enterprise Linux" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8.6 (Ootpa)" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:ID="rhel" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="fedora" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8.6" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.redhat.com/" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-07T19:32:02.979 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-08-07T19:32:02.980 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-07T19:32:02.980 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-07T19:32:02.980 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-07T19:32:02.980 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-07T19:32:02.980 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-08-07T19:32:02.998 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server 2023-08-07T19:32:02.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-08-07T19:32:02.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-07T19:32:03.062 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-08-07T19:32:03.129 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-08-07T19:32:03.129 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2023-08-07T19:32:03.187 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2023-08-07T19:32:03.187 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-07T19:32:03.245 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-08-07T19:32:03.246 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-07T19:32:03.246 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-07T19:32:03.246 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-07T19:32:03.246 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-07T19:32:03.246 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-08-07T19:32:03.274 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:32:03.365 INFO:teuthology.dispatcher.supervisor:Running job 7360677 2023-08-07T19:32:03.365 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_ce5f6d589a888dfc88977261bfa1ed63441de8fb/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-08-04_17:52:17-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7360677 --name rishabh-2023-08-04_17:52:17-fs-wip-rishabh-2023Aug1-3-testing-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/fs/misc}} -- /home/teuthworker/archive/rishabh-2023-08-04_17:52:17-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7360677/orig.config.yaml 2023-08-07T19:32:03.368 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-08-04_17:52:17-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7360677 2023-08-07T19:32:03.369 INFO:teuthology.dispatcher.supervisor:Job PID: 6082 2023-08-07T19:32:03.369 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-07T19:34:03.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:36:03.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:38:03.809 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:40:03.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:42:04.135 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:44:04.337 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:46:04.493 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:48:04.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:50:04.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:52:04.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:54:05.180 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:56:05.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T19:58:05.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:00:05.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:02:05.858 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:04:06.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:06:06.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:08:06.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:10:06.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:12:06.720 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:14:06.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:16:07.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:18:07.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:20:07.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:22:07.552 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:24:07.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:26:07.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:28:08.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:30:08.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:32:08.407 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:34:08.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:36:08.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:38:08.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:40:09.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:42:09.228 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:44:09.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:46:09.582 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:48:09.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:50:09.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:52:10.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:54:10.228 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:56:10.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T20:58:10.573 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:00:10.734 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:02:10.897 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:04:11.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:06:11.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:08:11.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:10:11.662 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:12:11.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:14:11.985 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:16:12.149 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:18:12.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:20:12.477 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:22:12.633 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:24:12.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:26:12.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:28:13.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:30:13.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:32:13.429 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:34:13.593 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:36:13.744 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:38:13.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:40:14.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:42:14.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:44:14.381 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:46:14.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:48:14.717 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:50:14.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:52:15.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:54:15.218 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:56:15.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T21:58:15.549 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T22:00:15.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T22:02:15.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T22:04:16.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-07T22:04:16.115 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-07T22:04:16.882 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-07T22:04:16.882 INFO:teuthology.nuke:Checking targets against current locks 2023-08-07T22:04:16.928 DEBUG:teuthology.nuke:shortname: smithi105 2023-08-07T22:04:16.929 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-07T22:04:16.955 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-04_17:52:17-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7360677', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-07 19:26:52.250396', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-07T22:04:16.958 DEBUG:teuthology.nuke:shortname: smithi146 2023-08-07T22:04:16.959 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-07T22:04:16.986 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-04_17:52:17-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7360677', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-07 19:26:52.251888', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-07T22:04:16.987 DEBUG:teuthology.nuke:shortname: smithi178 2023-08-07T22:04:16.988 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-07T22:04:17.012 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-04_17:52:17-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7360677', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-07 19:26:52.253169', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-07T22:04:17.022 INFO:teuthology.orchestra.console.smithi105:Power off 2023-08-07T22:04:17.022 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-07T22:04:17.065 INFO:teuthology.orchestra.console.smithi146:Power off 2023-08-07T22:04:17.065 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-07T22:04:17.073 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2023-08-07T22:04:17.073 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T22:04:17.094 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2023-08-07T22:04:17.107 INFO:teuthology.orchestra.console.smithi178:Power off 2023-08-07T22:04:17.108 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-07T22:04:17.116 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2023-08-07T22:04:17.117 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T22:04:17.130 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2023-08-07T22:04:17.130 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-07T22:04:17.141 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-08-07T22:04:17.165 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-08-07T22:04:17.195 INFO:teuthology.orchestra.console.smithi105:Power off completed 2023-08-07T22:04:17.243 INFO:teuthology.orchestra.console.smithi146:Power off completed 2023-08-07T22:04:17.267 INFO:teuthology.orchestra.console.smithi178:Power off completed 2023-08-07T22:04:17.361 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com 2023-08-07T22:04:17.404 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com 2023-08-07T22:04:17.452 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com