2023-11-15T11:42:06.122 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-15T11:42:06.124 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T11:42:06.267 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fa93b897b20>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fa93b63fca0>, signals=[15])
2023-11-15T11:42:06.268 INFO:teuthology.lock.ops:Start node 'smithi045.front.sepia.ceph.com' reimaging
2023-11-15T11:42:06.269 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: reset os type and version on server
2023-11-15T11:42:06.269 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server
2023-11-15T11:42:06.296 INFO:teuthology.lock.ops:Start node 'smithi135.front.sepia.ceph.com' reimaging
2023-11-15T11:42:06.297 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: reset os type and version on server
2023-11-15T11:42:06.297 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server
2023-11-15T11:42:06.299 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /host {"name": "smithi045"}
2023-11-15T11:42:06.320 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /host {"name": "smithi135"}
2023-11-15T11:42:06.379 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /image {"name": "smithi_ubuntu_22.04"}
2023-11-15T11:42:06.392 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /image {"name": "smithi_ubuntu_22.04"}
2023-11-15T11:42:06.424 DEBUG:teuthology.provision.fog.smithi045:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-11-15T11:42:06.424 DEBUG:teuthology.provision.fog.smithi045:FOG request: PUT /host/150 {"imageID": 6}
2023-11-15T11:42:06.432 DEBUG:teuthology.provision.fog.smithi135:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-11-15T11:42:06.433 DEBUG:teuthology.provision.fog.smithi135:FOG request: PUT /host/240 {"imageID": 6}
2023-11-15T11:42:06.476 INFO:teuthology.provision.fog.smithi135:Scheduling deploy of ubuntu 22.04
2023-11-15T11:42:06.477 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:42:06.480 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of ubuntu 22.04
2023-11-15T11:42:06.480 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:42:06.521 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /tasktype {"name": "deploy"}
2023-11-15T11:42:06.531 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /tasktype {"name": "deploy"}
2023-11-15T11:42:06.570 DEBUG:teuthology.provision.fog.smithi135:FOG request: POST /host/240/task {"taskTypeID": 1}
2023-11-15T11:42:06.573 DEBUG:teuthology.provision.fog.smithi045:FOG request: POST /host/150/task {"taskTypeID": 1}
2023-11-15T11:42:06.628 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:42:06.631 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:42:06.715 INFO:teuthology.orchestra.console.smithi045:Power off
2023-11-15T11:42:06.716 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-15T11:42:06.732 INFO:teuthology.orchestra.console.smithi135:Power off
2023-11-15T11:42:06.732 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-15T11:42:06.744 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off
2023-11-15T11:42:06.744 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:06.756 DEBUG:teuthology.orchestra.console.smithi135:power off output: Chassis Power Control: Down/Off
2023-11-15T11:42:06.756 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:06.768 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on
2023-11-15T11:42:06.779 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on
2023-11-15T11:42:10.774 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:10.781 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:10.883 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on
2023-11-15T11:42:10.891 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on
2023-11-15T11:42:14.886 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:14.893 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:14.995 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on
2023-11-15T11:42:15.002 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off
2023-11-15T11:42:15.103 INFO:teuthology.orchestra.console.smithi135:Power off completed
2023-11-15T11:42:15.204 INFO:teuthology.orchestra.console.smithi135:Power on
2023-11-15T11:42:15.204 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-15T11:42:15.226 DEBUG:teuthology.orchestra.console.smithi135:power on output: Chassis Power Control: Up/On
2023-11-15T11:42:15.226 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:15.252 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off
2023-11-15T11:42:18.998 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:19.107 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off
2023-11-15T11:42:19.208 INFO:teuthology.orchestra.console.smithi045:Power off completed
2023-11-15T11:42:19.253 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:19.310 INFO:teuthology.orchestra.console.smithi045:Power on
2023-11-15T11:42:19.311 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-15T11:42:19.334 DEBUG:teuthology.orchestra.console.smithi045:power on output: Chassis Power Control: Up/On
2023-11-15T11:42:19.334 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:19.359 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off
2023-11-15T11:42:19.362 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off
2023-11-15T11:42:23.363 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:23.370 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:23.472 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off
2023-11-15T11:42:23.479 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on
2023-11-15T11:42:23.580 INFO:teuthology.orchestra.console.smithi135:Power on completed
2023-11-15T11:42:23.681 INFO:teuthology.provision.fog.smithi135:Waiting for deploy to finish
2023-11-15T11:42:23.681 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:42:27.474 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T11:42:27.584 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on
2023-11-15T11:42:27.685 INFO:teuthology.orchestra.console.smithi045:Power on completed
2023-11-15T11:42:27.786 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish
2023-11-15T11:42:27.786 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:42:38.758 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:42:42.870 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:42:53.841 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:42:57.945 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:43:08.918 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:43:13.006 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:43:23.990 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:43:28.066 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:43:39.056 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:43:43.122 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:43:54.130 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:43:58.186 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:44:09.202 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:44:13.254 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:44:24.278 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:44:28.321 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:44:39.374 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:44:43.403 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:44:54.478 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:44:58.490 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:45:09.578 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:45:13.582 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:45:24.662 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:45:28.650 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None
2023-11-15T11:45:28.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:45:31.806 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45
2023-11-15T11:45:37.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:45:39.742 DEBUG:teuthology.provision.fog.smithi135:FOG request: GET /task/active None
2023-11-15T11:45:39.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:45:40.894 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45
2023-11-15T11:45:42.874 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:45:46.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:45:48.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:45:49.978 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45
2023-11-15T11:45:51.930 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:45:55.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:45:57.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:45:59.034 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45
2023-11-15T11:46:00.986 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:46:05.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:46:06.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:46:08.090 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45
2023-11-15T11:46:10.042 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:46:14.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:46:16.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:46:17.150 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45
2023-11-15T11:46:19.102 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:46:23.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:46:25.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:46:28.186 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:46:34.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:46:37.246 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:46:43.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:14.687 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:47:20.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:20.693 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:47:23.161 WARNING:teuthology.provision.fog.smithi045:timed out
2023-11-15T11:47:26.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:26.698 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:47:29.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:29.169 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45
2023-11-15T11:47:32.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:32.704 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2023-11-15T11:47:35.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:35.448 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-15T11:47:35.449 DEBUG:teuthology.orchestra.run.smithi045:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-15T11:47:35.992 INFO:teuthology.provision.fog.smithi045:Node is ready
2023-11-15T11:47:35.993 DEBUG:teuthology.orchestra.run.smithi045:> hostname
2023-11-15T11:47:36.040 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com
2023-11-15T11:47:36.041 DEBUG:teuthology.orchestra.run.smithi045:> grep smithi045.front.sepia.ceph.com /etc/hosts
2023-11-15T11:47:36.089 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045
2023-11-15T11:47:36.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname smithi045
2023-11-15T11:47:36.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/smithi045.front.sepia.ceph.com/smithi045/g' /etc/hosts
2023-11-15T11:47:36.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/172.21.15.45/172.21.15.45/g' /etc/hosts
2023-11-15T11:47:36.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/smithi045.front.sepia.ceph.com/smithi045/g' /etc/hostname
2023-11-15T11:47:36.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname smithi045
2023-11-15T11:47:36.422 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release
2023-11-15T11:47:36.468 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-11-15T11:47:36.468 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Ubuntu"
2023-11-15T11:47:36.468 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="22.04"
2023-11-15T11:47:36.468 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-11-15T11:47:36.469 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_CODENAME=jammy
2023-11-15T11:47:36.469 INFO:teuthology.orchestra.run.smithi045.stdout:ID=ubuntu
2023-11-15T11:47:36.469 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE=debian
2023-11-15T11:47:36.469 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.ubuntu.com/"
2023-11-15T11:47:36.470 INFO:teuthology.orchestra.run.smithi045.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-11-15T11:47:36.470 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-11-15T11:47:36.470 INFO:teuthology.orchestra.run.smithi045.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-11-15T11:47:36.470 INFO:teuthology.orchestra.run.smithi045.stdout:UBUNTU_CODENAME=jammy
2023-11-15T11:47:36.471 INFO:teuthology.provision.fog.smithi045:Deploy complete!
2023-11-15T11:47:36.472 DEBUG:teuthology.parallel:result is None
2023-11-15T11:47:38.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:39.018 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-15T11:47:39.018 DEBUG:teuthology.orchestra.run.smithi135:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-15T11:47:39.655 INFO:teuthology.provision.fog.smithi135:Node is ready
2023-11-15T11:47:39.656 DEBUG:teuthology.orchestra.run.smithi135:> hostname
2023-11-15T11:47:39.703 INFO:teuthology.orchestra.run.smithi135.stdout:smithi135.front.sepia.ceph.com
2023-11-15T11:47:39.703 DEBUG:teuthology.orchestra.run.smithi135:> grep smithi135.front.sepia.ceph.com /etc/hosts
2023-11-15T11:47:39.755 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135 smithi135.front.sepia.ceph.com smithi135
2023-11-15T11:47:39.757 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname smithi135
2023-11-15T11:47:39.916 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i -e 's/smithi135.front.sepia.ceph.com/smithi135/g' /etc/hosts
2023-11-15T11:47:39.929 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i -e 's/172.21.15.135/172.21.15.135/g' /etc/hosts
2023-11-15T11:47:39.982 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i -e 's/smithi135.front.sepia.ceph.com/smithi135/g' /etc/hostname
2023-11-15T11:47:40.038 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname smithi135
2023-11-15T11:47:40.097 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release
2023-11-15T11:47:40.147 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-11-15T11:47:40.147 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Ubuntu"
2023-11-15T11:47:40.147 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="22.04"
2023-11-15T11:47:40.148 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-11-15T11:47:40.148 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_CODENAME=jammy
2023-11-15T11:47:40.148 INFO:teuthology.orchestra.run.smithi135.stdout:ID=ubuntu
2023-11-15T11:47:40.148 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE=debian
2023-11-15T11:47:40.148 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.ubuntu.com/"
2023-11-15T11:47:40.149 INFO:teuthology.orchestra.run.smithi135.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-11-15T11:47:40.149 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-11-15T11:47:40.149 INFO:teuthology.orchestra.run.smithi135.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-11-15T11:47:40.149 INFO:teuthology.orchestra.run.smithi135.stdout:UBUNTU_CODENAME=jammy
2023-11-15T11:47:40.150 INFO:teuthology.provision.fog.smithi135:Deploy complete!
2023-11-15T11:47:40.150 DEBUG:teuthology.parallel:result is None
2023-11-15T11:47:40.332 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com
2023-11-15T11:47:40.332 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-15T11:47:40.357 INFO:teuthology.lock.ops:Checking smithi135.front.sepia.ceph.com
2023-11-15T11:47:40.357 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-15T11:47:40.375 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server
2023-11-15T11:47:40.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com'
2023-11-15T11:47:40.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:40.448 DEBUG:teuthology.orchestra.run.smithi045:> true
2023-11-15T11:47:40.560 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com'
2023-11-15T11:47:40.560 DEBUG:teuthology.orchestra.run.smithi045:> uname -m
2023-11-15T11:47:40.608 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64
2023-11-15T11:47:40.608 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release
2023-11-15T11:47:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-11-15T11:47:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Ubuntu"
2023-11-15T11:47:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="22.04"
2023-11-15T11:47:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-11-15T11:47:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_CODENAME=jammy
2023-11-15T11:47:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:ID=ubuntu
2023-11-15T11:47:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE=debian
2023-11-15T11:47:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.ubuntu.com/"
2023-11-15T11:47:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-11-15T11:47:40.658 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-11-15T11:47:40.658 INFO:teuthology.orchestra.run.smithi045.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-11-15T11:47:40.658 INFO:teuthology.orchestra.run.smithi045.stdout:UBUNTU_CODENAME=jammy
2023-11-15T11:47:40.659 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server
2023-11-15T11:47:40.682 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: set os type and version on server
2023-11-15T11:47:40.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com'
2023-11-15T11:47:40.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-15T11:47:40.756 DEBUG:teuthology.orchestra.run.smithi135:> true
2023-11-15T11:47:40.870 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com'
2023-11-15T11:47:40.871 DEBUG:teuthology.orchestra.run.smithi135:> uname -m
2023-11-15T11:47:40.919 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64
2023-11-15T11:47:40.919 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release
2023-11-15T11:47:40.966 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-11-15T11:47:40.967 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Ubuntu"
2023-11-15T11:47:40.967 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="22.04"
2023-11-15T11:47:40.967 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-11-15T11:47:40.967 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_CODENAME=jammy
2023-11-15T11:47:40.968 INFO:teuthology.orchestra.run.smithi135.stdout:ID=ubuntu
2023-11-15T11:47:40.968 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE=debian
2023-11-15T11:47:40.968 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.ubuntu.com/"
2023-11-15T11:47:40.968 INFO:teuthology.orchestra.run.smithi135.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-11-15T11:47:40.968 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-11-15T11:47:40.968 INFO:teuthology.orchestra.run.smithi135.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-11-15T11:47:40.969 INFO:teuthology.orchestra.run.smithi135.stdout:UBUNTU_CODENAME=jammy
2023-11-15T11:47:40.969 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server
2023-11-15T11:47:40.990 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T11:47:41.088 INFO:teuthology.dispatcher.supervisor:Running job 7459110
2023-11-15T11:47:41.088 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_rfriedma@teuthology --archive /home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459110 --name rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi --description rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache} -- /home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459110/orig.config.yaml
2023-11-15T11:47:41.091 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459110
2023-11-15T11:47:41.092 INFO:teuthology.dispatcher.supervisor:Job PID: 1928997
2023-11-15T11:47:41.092 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-15T11:49:41.190 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T11:51:41.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T11:53:41.528 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T11:55:41.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T11:57:41.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T11:59:42.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T12:01:42.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T12:03:42.377 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-15T12:03:42.456 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-11-15T12:03:42.618 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-15T12:03:42.619 INFO:teuthology.nuke:Checking targets against current locks
2023-11-15T12:03:42.674 DEBUG:teuthology.nuke:shortname: smithi045
2023-11-15T12:03:42.675 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-15T12:03:42.696 DEBUG:teuthology.nuke:shortname: smithi135
2023-11-15T12:03:42.696 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-15T12:03:42.700 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459110', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 11:42:05.153869', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG11CUxliKotiSKDvaE3YJITORtnYSsczwA4l6Myfq2/4lqGcsUR82roSkm3Pj2L0mY8kGjtuWPNZZ4BCFx63/I='}
2023-11-15T12:03:42.720 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459110', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 11:42:05.155819', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyAeVJN+ecVuy5ji3Q6Voclop2Pe8/GShsNgU7ERa7yCVBZozaYmCrZ5GwHpbb8NwRVJt3acXwdGC1ov22deMo='}
2023-11-15T12:03:42.758 INFO:teuthology.orchestra.console.smithi045:Power off
2023-11-15T12:03:42.759 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-15T12:03:42.781 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off
2023-11-15T12:03:42.782 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T12:03:42.795 INFO:teuthology.orchestra.console.smithi135:Power off
2023-11-15T12:03:42.796 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-15T12:03:42.806 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off
2023-11-15T12:03:42.817 DEBUG:teuthology.orchestra.console.smithi135:power off output: Chassis Power Control: Down/Off
2023-11-15T12:03:42.818 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-15T12:03:42.838 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off
2023-11-15T12:03:42.907 INFO:teuthology.orchestra.console.smithi045:Power off completed
2023-11-15T12:03:42.940 INFO:teuthology.orchestra.console.smithi135:Power off completed
2023-11-15T12:03:43.063 INFO:teuthology.lock.ops:unlocked: smithi045.front.sepia.ceph.com
2023-11-15T12:03:43.085 INFO:teuthology.lock.ops:unlocked: smithi135.front.sepia.ceph.com