2023-08-28T17:56:27.917 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc
2023-08-28T17:56:27.918 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-28T17:56:28.051 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f253800c9d0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f2538b171f0>, signals=[15])
2023-08-28T17:56:28.051 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging
2023-08-28T17:56:28.051 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server
2023-08-28T17:56:28.052 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2023-08-28T17:56:28.070 INFO:teuthology.lock.ops:Start node 'smithi120.front.sepia.ceph.com' reimaging
2023-08-28T17:56:28.071 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: reset os type and version on server
2023-08-28T17:56:28.071 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2023-08-28T17:56:28.072 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /host {"name": "smithi023"}
2023-08-28T17:56:28.095 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /host {"name": "smithi120"}
2023-08-28T17:56:28.145 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /image {"name": "smithi_ubuntu_22.04"}
2023-08-28T17:56:28.163 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /image {"name": "smithi_ubuntu_22.04"}
2023-08-28T17:56:28.186 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-08-28T17:56:28.186 DEBUG:teuthology.provision.fog.smithi023:FOG request: PUT /host/129 {"imageID": 6}
2023-08-28T17:56:28.202 DEBUG:teuthology.provision.fog.smithi120:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-08-28T17:56:28.202 DEBUG:teuthology.provision.fog.smithi120:FOG request: PUT /host/225 {"imageID": 6}
2023-08-28T17:56:28.244 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of ubuntu 22.04
2023-08-28T17:56:28.244 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:56:28.256 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of ubuntu 22.04
2023-08-28T17:56:28.256 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:56:28.360 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /tasktype {"name": "deploy"}
2023-08-28T17:56:28.367 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /tasktype {"name": "deploy"}
2023-08-28T17:56:28.391 DEBUG:teuthology.provision.fog.smithi120:FOG request: POST /host/225/task {"taskTypeID": 1}
2023-08-28T17:56:28.406 DEBUG:teuthology.provision.fog.smithi023:FOG request: POST /host/129/task {"taskTypeID": 1}
2023-08-28T17:56:28.452 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:56:28.470 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:56:28.628 INFO:teuthology.orchestra.console.smithi120:Power off
2023-08-28T17:56:28.628 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-28T17:56:28.639 INFO:teuthology.orchestra.console.smithi023:Power off
2023-08-28T17:56:28.639 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-28T17:56:28.647 DEBUG:teuthology.orchestra.console.smithi120:power off output: Chassis Power Control: Down/Off
2023-08-28T17:56:28.648 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T17:56:28.659 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off
2023-08-28T17:56:28.659 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T17:56:28.666 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is off
2023-08-28T17:56:28.678 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2023-08-28T17:56:28.767 INFO:teuthology.orchestra.console.smithi120:Power off completed
2023-08-28T17:56:28.779 INFO:teuthology.orchestra.console.smithi023:Power off completed
2023-08-28T17:56:28.868 INFO:teuthology.orchestra.console.smithi120:Power on
2023-08-28T17:56:28.868 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-28T17:56:28.879 INFO:teuthology.orchestra.console.smithi023:Power on
2023-08-28T17:56:28.880 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-28T17:56:28.897 DEBUG:teuthology.orchestra.console.smithi120:power on output: Chassis Power Control: Up/On
2023-08-28T17:56:28.897 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T17:56:28.910 DEBUG:teuthology.orchestra.console.smithi023:power on output: Chassis Power Control: Up/On
2023-08-28T17:56:28.910 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T17:56:28.917 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is off
2023-08-28T17:56:28.929 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2023-08-28T17:56:32.921 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T17:56:32.930 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T17:56:33.029 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is off
2023-08-28T17:56:33.037 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2023-08-28T17:56:37.034 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T17:56:37.041 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T17:56:37.143 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is on
2023-08-28T17:56:37.149 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2023-08-28T17:56:37.244 INFO:teuthology.orchestra.console.smithi120:Power on completed
2023-08-28T17:56:37.250 INFO:teuthology.orchestra.console.smithi023:Power on completed
2023-08-28T17:56:37.346 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish
2023-08-28T17:56:37.346 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:56:37.351 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish
2023-08-28T17:56:37.352 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:56:52.507 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:56:52.508 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:57:07.706 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:57:07.707 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:57:22.871 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:57:22.872 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:57:38.023 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:57:38.024 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:57:54.043 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:57:54.048 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:58:09.535 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:58:09.601 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:58:28.312 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:58:28.966 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:58:46.438 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:58:47.056 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:59:03.659 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:59:04.285 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:59:19.542 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:59:20.433 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:59:35.643 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:59:38.141 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T17:59:51.101 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T17:59:56.930 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T18:00:08.645 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T18:00:15.575 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T18:00:27.379 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-08-28T18:00:30.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:00:33.436 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:00:35.067 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T18:00:39.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:00:42.527 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:00:48.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:00:50.759 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T18:00:51.611 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:00:57.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:00.667 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:01:06.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:06.777 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-08-28T18:01:08.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:09.727 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:01:11.451 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:01:15.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:17.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:18.815 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:01:20.507 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:01:24.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:26.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:27.899 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:01:29.563 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:01:33.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:35.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:36.955 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:01:38.619 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:01:42.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:44.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:46.015 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:01:47.675 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:01:52.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:53.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:01:55.099 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:01:56.732 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:02:01.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:02:02.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:02:34.172 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:02:40.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:02:40.180 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:02:46.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:02:46.185 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:02:52.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:02:52.193 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120
2023-08-28T18:02:58.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:02:58.497 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-28T18:02:58.498 DEBUG:teuthology.orchestra.run.smithi120:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-28T18:02:59.326 INFO:teuthology.provision.fog.smithi120:Node is ready
2023-08-28T18:02:59.326 DEBUG:teuthology.orchestra.run.smithi120:> hostname
2023-08-28T18:02:59.373 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com
2023-08-28T18:02:59.373 DEBUG:teuthology.orchestra.run.smithi120:> grep smithi120.front.sepia.ceph.com /etc/hosts
2023-08-28T18:02:59.420 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120
2023-08-28T18:02:59.421 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname smithi120
2023-08-28T18:02:59.573 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/smithi120.front.sepia.ceph.com/smithi120/g' /etc/hosts
2023-08-28T18:02:59.586 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/172.21.15.120/172.21.15.120/g' /etc/hosts
2023-08-28T18:02:59.641 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/smithi120.front.sepia.ceph.com/smithi120/g' /etc/hostname
2023-08-28T18:02:59.695 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname smithi120
2023-08-28T18:02:59.750 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release
2023-08-28T18:02:59.800 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-08-28T18:02:59.800 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Ubuntu"
2023-08-28T18:02:59.800 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="22.04"
2023-08-28T18:02:59.800 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-08-28T18:02:59.800 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_CODENAME=jammy
2023-08-28T18:02:59.800 INFO:teuthology.orchestra.run.smithi120.stdout:ID=ubuntu
2023-08-28T18:02:59.801 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE=debian
2023-08-28T18:02:59.801 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.ubuntu.com/"
2023-08-28T18:02:59.801 INFO:teuthology.orchestra.run.smithi120.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-08-28T18:02:59.801 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-08-28T18:02:59.801 INFO:teuthology.orchestra.run.smithi120.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-08-28T18:02:59.801 INFO:teuthology.orchestra.run.smithi120.stdout:UBUNTU_CODENAME=jammy
2023-08-28T18:02:59.801 INFO:teuthology.provision.fog.smithi120:Deploy complete!
2023-08-28T18:02:59.802 DEBUG:teuthology.parallel:result is None
2023-08-28T18:03:01.107 WARNING:teuthology.provision.fog.smithi023:timed out
2023-08-28T18:03:07.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:03:08.124 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-08-28T18:03:14.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:03:14.345 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-28T18:03:14.345 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-28T18:03:14.814 INFO:teuthology.provision.fog.smithi023:Node is ready
2023-08-28T18:03:14.814 DEBUG:teuthology.orchestra.run.smithi023:> hostname
2023-08-28T18:03:14.860 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com
2023-08-28T18:03:14.860 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts
2023-08-28T18:03:14.924 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023
2023-08-28T18:03:14.926 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2023-08-28T18:03:15.065 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts
2023-08-28T18:03:15.079 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts
2023-08-28T18:03:15.137 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname
2023-08-28T18:03:15.196 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2023-08-28T18:03:15.255 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2023-08-28T18:03:15.308 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="Ubuntu"
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="22.04"
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_CODENAME=jammy
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:ID=ubuntu
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE=debian
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://www.ubuntu.com/"
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-08-28T18:03:15.309 INFO:teuthology.orchestra.run.smithi023.stdout:UBUNTU_CODENAME=jammy
2023-08-28T18:03:15.310 INFO:teuthology.provision.fog.smithi023:Deploy complete!
2023-08-28T18:03:15.310 DEBUG:teuthology.parallel:result is None
2023-08-28T18:03:15.506 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com
2023-08-28T18:03:15.506 INFO:teuthology.lock.ops:New key found. Updating...
2023-08-28T18:03:15.526 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com
2023-08-28T18:03:15.526 INFO:teuthology.lock.ops:New key found. Updating...
2023-08-28T18:03:15.553 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server
2023-08-28T18:03:15.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com'
2023-08-28T18:03:15.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:03:15.631 DEBUG:teuthology.orchestra.run.smithi023:> true
2023-08-28T18:03:15.723 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com'
2023-08-28T18:03:15.724 DEBUG:teuthology.orchestra.run.smithi023:> uname -m
2023-08-28T18:03:15.776 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64
2023-08-28T18:03:15.776 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="Ubuntu"
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="22.04"
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_CODENAME=jammy
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:ID=ubuntu
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE=debian
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://www.ubuntu.com/"
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-08-28T18:03:15.824 INFO:teuthology.orchestra.run.smithi023.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-08-28T18:03:15.825 INFO:teuthology.orchestra.run.smithi023.stdout:UBUNTU_CODENAME=jammy
2023-08-28T18:03:15.825 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2023-08-28T18:03:15.843 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server
2023-08-28T18:03:15.843 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2023-08-28T18:03:15.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-28T18:03:15.917 DEBUG:teuthology.orchestra.run.smithi120:> true
2023-08-28T18:03:16.155 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com'
2023-08-28T18:03:16.155 DEBUG:teuthology.orchestra.run.smithi120:> uname -m
2023-08-28T18:03:16.203 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64
2023-08-28T18:03:16.203 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release
2023-08-28T18:03:16.252 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-08-28T18:03:16.252 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Ubuntu"
2023-08-28T18:03:16.252 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="22.04"
2023-08-28T18:03:16.252 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-08-28T18:03:16.252 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_CODENAME=jammy
2023-08-28T18:03:16.252 INFO:teuthology.orchestra.run.smithi120.stdout:ID=ubuntu
2023-08-28T18:03:16.253 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE=debian
2023-08-28T18:03:16.253 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.ubuntu.com/"
2023-08-28T18:03:16.253 INFO:teuthology.orchestra.run.smithi120.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-08-28T18:03:16.253 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-08-28T18:03:16.253 INFO:teuthology.orchestra.run.smithi120.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-08-28T18:03:16.253 INFO:teuthology.orchestra.run.smithi120.stdout:UBUNTU_CODENAME=jammy
2023-08-28T18:03:16.253 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2023-08-28T18:03:16.368 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-28T18:03:16.455 INFO:teuthology.dispatcher.supervisor:Running job 7382277
2023-08-28T18:03:16.455 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/virtualenv/bin/teuthology -v --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-08-28_15:00:18-rados:thrash-wip-matanb-trim-61962-distro-default-smithi/7382277 --name matan-2023-08-28_15:00:18-rados:thrash-wip-matanb-trim-61962-distro-default-smithi --description rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} -- /home/teuthworker/archive/matan-2023-08-28_15:00:18-rados:thrash-wip-matanb-trim-61962-distro-default-smithi/7382277/orig.config.yaml
2023-08-28T18:03:16.457 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/matan-2023-08-28_15:00:18-rados:thrash-wip-matanb-trim-61962-distro-default-smithi/7382277
2023-08-28T18:03:16.458 INFO:teuthology.dispatcher.supervisor:Job PID: 17857
2023-08-28T18:03:16.458 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-08-28T18:05:16.555 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-28T18:05:16.707 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/matan-2023-08-28_15:00:18-rados:thrash-wip-matanb-trim-61962-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-08-28T18:05:18.276 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-08-28T18:05:18.389 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-08-28T18:05:18.390 INFO:teuthology.nuke:Checking targets against current locks
2023-08-28T18:05:18.429 DEBUG:teuthology.nuke:shortname: smithi023
2023-08-28T18:05:18.429 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-28T18:05:18.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi023.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-08-28_15:00:18-rados:thrash-wip-matanb-trim-61962-distro-default-smithi/7382277', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-28 17:56:27.133613', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINK6gQ8ORtjIyRaU3nTGsw45SIeLM4NNq9We6tLj296B'}
2023-08-28T18:05:18.454 DEBUG:teuthology.nuke:shortname: smithi120
2023-08-28T18:05:18.454 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-28T18:05:18.475 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-08-28_15:00:18-rados:thrash-wip-matanb-trim-61962-distro-default-smithi/7382277', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-28 17:56:27.132180', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINUD18spF3e144eUoesKrH2wVHaNoszeDmVi202NaQX2'}
2023-08-28T18:05:18.497 INFO:teuthology.orchestra.console.smithi023:Power off
2023-08-28T18:05:18.497 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-28T18:05:18.518 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off
2023-08-28T18:05:18.518 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T18:05:18.529 INFO:teuthology.orchestra.console.smithi120:Power off
2023-08-28T18:05:18.529 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-28T18:05:18.539 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2023-08-28T18:05:18.547 DEBUG:teuthology.orchestra.console.smithi120:power off output: Chassis Power Control: Down/Off
2023-08-28T18:05:18.547 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-28T18:05:18.569 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is off
2023-08-28T18:05:18.641 INFO:teuthology.orchestra.console.smithi023:Power off completed
2023-08-28T18:05:18.670 INFO:teuthology.orchestra.console.smithi120:Power off completed
2023-08-28T18:05:18.790 INFO:teuthology.lock.ops:unlocked: smithi023.front.sepia.ceph.com
2023-08-28T18:05:18.808 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com