2023-10-25T07:34:21.687 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-25T07:34:21.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:34:21.856 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2d00694dc0>, signals=[15]) 2023-10-25T07:34:21.858 INFO:teuthology.lock.ops:Start node 'smithi003.front.sepia.ceph.com' reimaging 2023-10-25T07:34:21.858 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: reset os type and version on server 2023-10-25T07:34:21.859 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-10-25T07:34:21.880 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging 2023-10-25T07:34:21.881 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server 2023-10-25T07:34:21.881 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-10-25T07:34:21.883 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /host {"name": "smithi003"} 2023-10-25T07:34:21.905 INFO:teuthology.lock.ops:Start node 'smithi071.front.sepia.ceph.com' reimaging 2023-10-25T07:34:21.906 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: reset os type and version on server 2023-10-25T07:34:21.906 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2023-10-25T07:34:21.907 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /host {"name": "smithi062"} 2023-10-25T07:34:21.930 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /host {"name": "smithi071"} 2023-10-25T07:34:21.957 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-10-25T07:34:21.976 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-10-25T07:34:21.997 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-10-25T07:34:22.000 DEBUG:teuthology.provision.fog.smithi003:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-10-25T07:34:22.001 DEBUG:teuthology.provision.fog.smithi003:FOG request: PUT /host/109 {"imageID": 2} 2023-10-25T07:34:22.025 DEBUG:teuthology.provision.fog.smithi062:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-10-25T07:34:22.026 DEBUG:teuthology.provision.fog.smithi062:FOG request: PUT /host/167 {"imageID": 2} 2023-10-25T07:34:22.033 DEBUG:teuthology.provision.fog.smithi071:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-10-25T07:34:22.034 DEBUG:teuthology.provision.fog.smithi071:FOG request: PUT /host/176 {"imageID": 2} 2023-10-25T07:34:22.049 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of ubuntu 20.04 2023-10-25T07:34:22.049 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:34:22.085 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of ubuntu 20.04 2023-10-25T07:34:22.086 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:34:22.089 INFO:teuthology.provision.fog.smithi071:Scheduling deploy of ubuntu 20.04 2023-10-25T07:34:22.089 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:34:22.113 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /tasktype {"name": "deploy"} 2023-10-25T07:34:22.146 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /tasktype {"name": "deploy"} 2023-10-25T07:34:22.150 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /tasktype {"name": "deploy"} 2023-10-25T07:34:22.151 DEBUG:teuthology.provision.fog.smithi003:FOG request: POST /host/109/task {"taskTypeID": 1} 2023-10-25T07:34:22.202 DEBUG:teuthology.provision.fog.smithi071:FOG request: POST /host/176/task {"taskTypeID": 1} 2023-10-25T07:34:22.204 DEBUG:teuthology.provision.fog.smithi062:FOG request: POST /host/167/task {"taskTypeID": 1} 2023-10-25T07:34:22.207 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:34:22.252 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:34:22.277 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:34:22.327 INFO:teuthology.orchestra.console.smithi003:Power off 2023-10-25T07:34:22.328 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-25T07:34:22.351 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-10-25T07:34:22.352 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:22.365 INFO:teuthology.orchestra.console.smithi062:Power off 2023-10-25T07:34:22.366 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-25T07:34:22.378 INFO:teuthology.orchestra.console.smithi071:Power off 2023-10-25T07:34:22.378 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-25T07:34:22.385 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-25T07:34:22.387 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2023-10-25T07:34:22.388 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:22.399 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2023-10-25T07:34:22.400 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:22.408 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-10-25T07:34:22.422 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-10-25T07:34:26.386 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:26.410 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:26.423 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:26.495 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-25T07:34:26.518 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-10-25T07:34:26.532 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-10-25T07:34:30.498 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:30.519 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:30.533 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:30.607 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-25T07:34:30.628 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-10-25T07:34:30.641 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2023-10-25T07:34:30.709 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-10-25T07:34:30.742 INFO:teuthology.orchestra.console.smithi071:Power off completed 2023-10-25T07:34:30.810 INFO:teuthology.orchestra.console.smithi003:Power on 2023-10-25T07:34:30.810 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-25T07:34:30.830 DEBUG:teuthology.orchestra.console.smithi003:power on output: Chassis Power Control: Up/On 2023-10-25T07:34:30.831 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:30.844 INFO:teuthology.orchestra.console.smithi071:Power on 2023-10-25T07:34:30.844 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-25T07:34:30.852 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-25T07:34:30.865 DEBUG:teuthology.orchestra.console.smithi071:power on output: Chassis Power Control: Up/On 2023-10-25T07:34:30.865 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:30.885 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2023-10-25T07:34:34.632 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:34.741 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-10-25T07:34:34.842 INFO:teuthology.orchestra.console.smithi062:Power off completed 2023-10-25T07:34:34.854 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:34.886 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:34.943 INFO:teuthology.orchestra.console.smithi062:Power on 2023-10-25T07:34:34.944 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-25T07:34:34.966 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-25T07:34:34.970 DEBUG:teuthology.orchestra.console.smithi062:power on output: Chassis Power Control: Up/On 2023-10-25T07:34:34.971 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:34.994 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-10-25T07:34:34.996 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2023-10-25T07:34:38.971 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:38.995 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:39.003 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T07:34:39.081 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-25T07:34:39.104 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-10-25T07:34:39.111 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-10-25T07:34:39.183 INFO:teuthology.orchestra.console.smithi003:Power on completed 2023-10-25T07:34:39.205 INFO:teuthology.orchestra.console.smithi062:Power on completed 2023-10-25T07:34:39.212 INFO:teuthology.orchestra.console.smithi071:Power on completed 2023-10-25T07:34:39.284 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish 2023-10-25T07:34:39.284 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:34:39.306 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish 2023-10-25T07:34:39.307 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:34:39.313 INFO:teuthology.provision.fog.smithi071:Waiting for deploy to finish 2023-10-25T07:34:39.313 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:34:54.390 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:34:54.392 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:34:54.401 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:35:09.490 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:35:09.492 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:35:09.492 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:35:24.582 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:35:24.591 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:35:24.593 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:35:39.674 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:35:39.676 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:35:39.677 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:35:54.786 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:35:54.789 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:35:54.791 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:36:09.910 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:36:09.912 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:36:09.913 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:36:25.018 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:36:25.020 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:36:25.021 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:36:40.134 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:36:40.136 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:36:40.137 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:36:55.250 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:36:55.251 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:36:55.252 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:37:10.366 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:37:10.368 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:37:10.369 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:37:25.474 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:37:25.476 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:37:25.477 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:37:40.582 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:37:40.584 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:37:40.585 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:37:55.690 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:37:55.692 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:37:55.693 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:38:10.795 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:38:10.796 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:38:10.800 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-25T07:38:10.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:25.883 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-10-25T07:38:25.884 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-10-25T07:38:25.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:25.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:29.370 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-25T07:38:35.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:38.526 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-25T07:38:44.474 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-25T07:38:44.478 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-10-25T07:38:44.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:47.610 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-25T07:38:50.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:50.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:53.533 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-25T07:38:53.534 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-10-25T07:38:53.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:56.670 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-25T07:38:59.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:38:59.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:39:02.622 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-25T07:39:02.623 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-10-25T07:39:02.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:39:08.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:39:08.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:39:11.706 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-25T07:39:11.707 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-10-25T07:39:17.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:39:17.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:39:20.762 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-25T07:39:26.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:39:49.846 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-25T07:39:49.847 DEBUG:teuthology.orchestra.run.smithi071:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-25T07:39:58.911 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-25T07:39:58.912 DEBUG:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-25T07:40:02.674 WARNING:teuthology.provision.fog.smithi003:timed out 2023-10-25T07:40:08.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:40:08.881 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-25T07:40:08.882 DEBUG:teuthology.orchestra.run.smithi003:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-25T07:40:09.318 INFO:teuthology.provision.fog.smithi003:Node is ready 2023-10-25T07:40:09.318 DEBUG:teuthology.orchestra.run.smithi003:> hostname 2023-10-25T07:40:09.325 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com 2023-10-25T07:40:09.325 DEBUG:teuthology.orchestra.run.smithi003:> grep smithi003.front.sepia.ceph.com /etc/hosts 2023-10-25T07:40:09.372 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003 2023-10-25T07:40:09.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003 2023-10-25T07:40:09.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hosts 2023-10-25T07:40:09.480 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/172.21.15.3/172.21.15.3/g' /etc/hosts 2023-10-25T07:40:09.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hostname 2023-10-25T07:40:09.510 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003 2023-10-25T07:40:09.522 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-10-25T07:40:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Ubuntu" 2023-10-25T07:40:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-25T07:40:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:ID=ubuntu 2023-10-25T07:40:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE=debian 2023-10-25T07:40:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-25T07:40:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="20.04" 2023-10-25T07:40:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-25T07:40:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-25T07:40:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-25T07:40:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-25T07:40:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_CODENAME=focal 2023-10-25T07:40:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:UBUNTU_CODENAME=focal 2023-10-25T07:40:09.530 INFO:teuthology.provision.fog.smithi003:Deploy complete! 2023-10-25T07:40:09.531 DEBUG:teuthology.parallel:result is None 2023-10-25T07:40:11.044 INFO:teuthology.provision.fog.smithi071:Node is ready 2023-10-25T07:40:11.045 DEBUG:teuthology.orchestra.run.smithi071:> hostname 2023-10-25T07:40:11.051 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071.front.sepia.ceph.com 2023-10-25T07:40:11.052 DEBUG:teuthology.orchestra.run.smithi071:> grep smithi071.front.sepia.ceph.com /etc/hosts 2023-10-25T07:40:11.101 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71 smithi071.front.sepia.ceph.com smithi071 2023-10-25T07:40:11.101 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname smithi071 2023-10-25T07:40:11.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/smithi071.front.sepia.ceph.com/smithi071/g' /etc/hosts 2023-10-25T07:40:11.207 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/172.21.15.71/172.21.15.71/g' /etc/hosts 2023-10-25T07:40:11.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/smithi071.front.sepia.ceph.com/smithi071/g' /etc/hostname 2023-10-25T07:40:11.277 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname smithi071 2023-10-25T07:40:11.287 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2023-10-25T07:40:11.294 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Ubuntu" 2023-10-25T07:40:11.294 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-25T07:40:11.294 INFO:teuthology.orchestra.run.smithi071.stdout:ID=ubuntu 2023-10-25T07:40:11.294 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE=debian 2023-10-25T07:40:11.295 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-25T07:40:11.295 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="20.04" 2023-10-25T07:40:11.295 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-25T07:40:11.295 INFO:teuthology.orchestra.run.smithi071.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-25T07:40:11.295 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-25T07:40:11.296 INFO:teuthology.orchestra.run.smithi071.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-25T07:40:11.296 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_CODENAME=focal 2023-10-25T07:40:11.296 INFO:teuthology.orchestra.run.smithi071.stdout:UBUNTU_CODENAME=focal 2023-10-25T07:40:11.296 INFO:teuthology.provision.fog.smithi071:Deploy complete! 2023-10-25T07:40:11.297 DEBUG:teuthology.parallel:result is None 2023-10-25T07:40:19.674 INFO:teuthology.provision.fog.smithi062:Node is ready 2023-10-25T07:40:19.675 DEBUG:teuthology.orchestra.run.smithi062:> hostname 2023-10-25T07:40:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com 2023-10-25T07:40:19.682 DEBUG:teuthology.orchestra.run.smithi062:> grep smithi062.front.sepia.ceph.com /etc/hosts 2023-10-25T07:40:19.730 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062 2023-10-25T07:40:19.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2023-10-25T07:40:19.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hosts 2023-10-25T07:40:19.825 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/172.21.15.62/172.21.15.62/g' /etc/hosts 2023-10-25T07:40:19.883 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hostname 2023-10-25T07:40:19.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2023-10-25T07:40:19.904 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-10-25T07:40:19.952 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Ubuntu" 2023-10-25T07:40:19.952 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-25T07:40:19.952 INFO:teuthology.orchestra.run.smithi062.stdout:ID=ubuntu 2023-10-25T07:40:19.953 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE=debian 2023-10-25T07:40:19.953 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-25T07:40:19.953 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="20.04" 2023-10-25T07:40:19.953 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-25T07:40:19.954 INFO:teuthology.orchestra.run.smithi062.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-25T07:40:19.954 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-25T07:40:19.954 INFO:teuthology.orchestra.run.smithi062.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-25T07:40:19.954 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_CODENAME=focal 2023-10-25T07:40:19.955 INFO:teuthology.orchestra.run.smithi062.stdout:UBUNTU_CODENAME=focal 2023-10-25T07:40:19.955 INFO:teuthology.provision.fog.smithi062:Deploy complete! 2023-10-25T07:40:19.956 DEBUG:teuthology.parallel:result is None 2023-10-25T07:40:20.282 INFO:teuthology.lock.ops:Checking smithi003.front.sepia.ceph.com 2023-10-25T07:40:20.283 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-25T07:40:20.304 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com 2023-10-25T07:40:20.305 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-25T07:40:20.321 INFO:teuthology.lock.ops:Checking smithi071.front.sepia.ceph.com 2023-10-25T07:40:20.321 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-25T07:40:20.347 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: set os type and version on server 2023-10-25T07:40:20.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-25T07:40:20.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:40:20.398 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-10-25T07:40:20.519 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-25T07:40:20.520 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2023-10-25T07:40:20.526 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2023-10-25T07:40:20.527 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-10-25T07:40:20.534 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Ubuntu" 2023-10-25T07:40:20.534 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-25T07:40:20.534 INFO:teuthology.orchestra.run.smithi003.stdout:ID=ubuntu 2023-10-25T07:40:20.535 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE=debian 2023-10-25T07:40:20.536 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-25T07:40:20.536 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="20.04" 2023-10-25T07:40:20.536 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-25T07:40:20.536 INFO:teuthology.orchestra.run.smithi003.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-25T07:40:20.536 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-25T07:40:20.537 INFO:teuthology.orchestra.run.smithi003.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-25T07:40:20.537 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_CODENAME=focal 2023-10-25T07:40:20.537 INFO:teuthology.orchestra.run.smithi003.stdout:UBUNTU_CODENAME=focal 2023-10-25T07:40:20.537 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-10-25T07:40:20.563 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server 2023-10-25T07:40:20.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-25T07:40:20.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:40:20.613 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-10-25T07:40:20.740 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-25T07:40:20.740 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-10-25T07:40:20.747 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-10-25T07:40:20.747 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-10-25T07:40:20.753 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Ubuntu" 2023-10-25T07:40:20.754 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-25T07:40:20.754 INFO:teuthology.orchestra.run.smithi062.stdout:ID=ubuntu 2023-10-25T07:40:20.754 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE=debian 2023-10-25T07:40:20.754 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-25T07:40:20.755 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="20.04" 2023-10-25T07:40:20.755 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-25T07:40:20.755 INFO:teuthology.orchestra.run.smithi062.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-25T07:40:20.755 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-25T07:40:20.755 INFO:teuthology.orchestra.run.smithi062.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-25T07:40:20.756 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_CODENAME=focal 2023-10-25T07:40:20.756 INFO:teuthology.orchestra.run.smithi062.stdout:UBUNTU_CODENAME=focal 2023-10-25T07:40:20.757 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-10-25T07:40:20.779 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: set os type and version on server 2023-10-25T07:40:20.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-25T07:40:20.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-25T07:40:20.828 DEBUG:teuthology.orchestra.run.smithi071:> true 2023-10-25T07:40:20.943 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-25T07:40:20.944 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2023-10-25T07:40:20.950 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2023-10-25T07:40:20.951 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2023-10-25T07:40:20.957 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Ubuntu" 2023-10-25T07:40:20.957 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-25T07:40:20.957 INFO:teuthology.orchestra.run.smithi071.stdout:ID=ubuntu 2023-10-25T07:40:20.958 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE=debian 2023-10-25T07:40:20.958 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-25T07:40:20.958 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="20.04" 2023-10-25T07:40:20.958 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-25T07:40:20.958 INFO:teuthology.orchestra.run.smithi071.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-25T07:40:20.959 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-25T07:40:20.959 INFO:teuthology.orchestra.run.smithi071.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-25T07:40:20.959 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_CODENAME=focal 2023-10-25T07:40:20.959 INFO:teuthology.orchestra.run.smithi071.stdout:UBUNTU_CODENAME=focal 2023-10-25T07:40:20.960 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2023-10-25T07:40:20.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:40:21.081 INFO:teuthology.dispatcher.supervisor:Running job 7436847 2023-10-25T07:40:21.081 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/virtualenv/bin/teuthology -v --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-10-25_07:18:31-upgrade:quincy-p2p:quincy-p2p-stress-split-main-distro-default-smithi/7436847 --name matan-2023-10-25_07:18:31-upgrade:quincy-p2p:quincy-p2p-stress-split-main-distro-default-smithi --description upgrade:quincy-p2p:quincy-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/quincy 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 6-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} -- /home/teuthworker/archive/matan-2023-10-25_07:18:31-upgrade:quincy-p2p:quincy-p2p-stress-split-main-distro-default-smithi/7436847/orig.config.yaml 2023-10-25T07:40:21.084 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/matan-2023-10-25_07:18:31-upgrade:quincy-p2p:quincy-p2p-stress-split-main-distro-default-smithi/7436847 2023-10-25T07:40:21.085 INFO:teuthology.dispatcher.supervisor:Job PID: 3879673 2023-10-25T07:40:21.085 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-25T07:42:21.160 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:44:21.331 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:46:21.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:48:21.671 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:50:21.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:52:22.027 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:54:22.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:56:22.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T07:58:22.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:00:22.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:02:22.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:04:23.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:06:23.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:08:23.294 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:10:23.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:12:23.634 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:14:23.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:16:23.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:18:24.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:20:24.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:22:24.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:24:24.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:26:24.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:28:24.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:30:25.135 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:32:25.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:34:25.469 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:36:25.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:38:25.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:40:25.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:42:26.143 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:44:26.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:46:26.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:48:26.628 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:50:26.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:52:26.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:54:27.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:56:27.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:58:27.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-25T08:58:27.512 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-25T08:58:27.739 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-25T08:58:27.739 INFO:teuthology.nuke:Checking targets against current locks 2023-10-25T08:58:27.784 DEBUG:teuthology.nuke:shortname: smithi003 2023-10-25T08:58:27.784 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-25T08:58:27.810 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-10-25_07:18:31-upgrade:quincy-p2p:quincy-p2p-stress-split-main-distro-default-smithi/7436847', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-25 07:34:20.829042', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-25T08:58:27.813 DEBUG:teuthology.nuke:shortname: smithi062 2023-10-25T08:58:27.813 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-25T08:58:27.838 DEBUG:teuthology.nuke:shortname: smithi071 2023-10-25T08:58:27.838 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-25T08:58:27.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-10-25_07:18:31-upgrade:quincy-p2p:quincy-p2p-stress-split-main-distro-default-smithi/7436847', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-25 07:34:20.827417', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-25T08:58:27.861 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-10-25_07:18:31-upgrade:quincy-p2p:quincy-p2p-stress-split-main-distro-default-smithi/7436847', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-25 07:34:20.829974', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-25T08:58:27.870 INFO:teuthology.orchestra.console.smithi003:Power off 2023-10-25T08:58:27.871 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-25T08:58:27.892 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-10-25T08:58:27.893 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T08:58:27.914 INFO:teuthology.orchestra.console.smithi062:Power off 2023-10-25T08:58:27.915 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-25T08:58:27.926 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-25T08:58:27.937 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2023-10-25T08:58:27.937 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T08:58:27.951 INFO:teuthology.orchestra.console.smithi071:Power off 2023-10-25T08:58:27.951 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-25T08:58:27.960 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-10-25T08:58:27.972 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2023-10-25T08:58:27.972 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-25T08:58:27.993 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2023-10-25T08:58:28.027 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-10-25T08:58:28.061 INFO:teuthology.orchestra.console.smithi062:Power off completed 2023-10-25T08:58:28.095 INFO:teuthology.orchestra.console.smithi071:Power off completed 2023-10-25T08:58:28.178 INFO:teuthology.lock.ops:unlocked: smithi003.front.sepia.ceph.com 2023-10-25T08:58:28.213 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com 2023-10-25T08:58:28.244 INFO:teuthology.lock.ops:unlocked: smithi071.front.sepia.ceph.com