2023-12-14T20:18:53.632 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-14T20:18:53.633 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:18:53.751 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fbb342ae280>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fbb3520b940>, signals=[15])
2023-12-14T20:18:53.751 INFO:teuthology.lock.ops:Start node 'smithi037.front.sepia.ceph.com' reimaging
2023-12-14T20:18:53.751 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: reset os type and version on server
2023-12-14T20:18:53.752 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server
2023-12-14T20:18:53.774 INFO:teuthology.lock.ops:Start node 'smithi053.front.sepia.ceph.com' reimaging
2023-12-14T20:18:53.774 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: reset os type and version on server
2023-12-14T20:18:53.774 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server
2023-12-14T20:18:53.882 DEBUG:teuthology.provision.fog.smithi037:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-12-14T20:18:53.904 DEBUG:teuthology.provision.fog.smithi053:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-12-14T20:18:53.928 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of ubuntu 22.04
2023-12-14T20:18:53.953 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of ubuntu 22.04
2023-12-14T20:18:54.317 INFO:teuthology.orchestra.console.smithi037:Power off
2023-12-14T20:18:54.317 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-14T20:18:54.330 INFO:teuthology.orchestra.console.smithi053:Power off
2023-12-14T20:18:54.330 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-14T20:18:54.339 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off
2023-12-14T20:18:54.339 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T20:18:54.351 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off
2023-12-14T20:18:54.351 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T20:18:54.359 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off
2023-12-14T20:18:54.371 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off
2023-12-14T20:18:54.460 INFO:teuthology.orchestra.console.smithi037:Power off completed
2023-12-14T20:18:54.471 INFO:teuthology.orchestra.console.smithi053:Power off completed
2023-12-14T20:18:54.561 INFO:teuthology.orchestra.console.smithi037:Power on
2023-12-14T20:18:54.561 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-14T20:18:54.572 INFO:teuthology.orchestra.console.smithi053:Power on
2023-12-14T20:18:54.573 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-14T20:18:54.582 DEBUG:teuthology.orchestra.console.smithi037:power on output: Chassis Power Control: Up/On
2023-12-14T20:18:54.583 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T20:18:54.593 DEBUG:teuthology.orchestra.console.smithi053:power on output: Chassis Power Control: Up/On
2023-12-14T20:18:54.594 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T20:18:54.603 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off
2023-12-14T20:18:54.617 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off
2023-12-14T20:18:58.605 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T20:18:58.617 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T20:18:58.714 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off
2023-12-14T20:18:58.725 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off
2023-12-14T20:19:02.719 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T20:19:02.725 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T20:19:02.827 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on
2023-12-14T20:19:02.832 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on
2023-12-14T20:19:02.929 INFO:teuthology.orchestra.console.smithi037:Power on completed
2023-12-14T20:19:02.933 INFO:teuthology.orchestra.console.smithi053:Power on completed
2023-12-14T20:19:03.029 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish
2023-12-14T20:19:03.033 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish
2023-12-14T20:23:20.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:23:20.905 ERROR:teuthology.orchestra.connection:Error authenticating with smithi037.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-14T20:23:35.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:23:38.901 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-12-14T20:23:44.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:23:47.953 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-12-14T20:23:53.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:23:57.009 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-12-14T20:24:03.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:24:06.065 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-12-14T20:24:12.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:24:15.121 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-12-14T20:24:21.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:24:24.908 WARNING:teuthology.provision.fog.smithi037:timed out
2023-12-14T20:24:30.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:24:49.425 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37
2023-12-14T20:24:55.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:24:55.428 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37
2023-12-14T20:25:01.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:25:01.431 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37
2023-12-14T20:25:07.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:25:07.435 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37
2023-12-14T20:25:13.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:25:16.466 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37
2023-12-14T20:25:21.125 WARNING:teuthology.provision.fog.smithi053:timed out
2023-12-14T20:25:22.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:25:22.660 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-14T20:25:22.661 DEBUG:teuthology.orchestra.run.smithi037:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-14T20:25:23.089 INFO:teuthology.provision.fog.smithi037:Node is ready
2023-12-14T20:25:23.089 DEBUG:teuthology.orchestra.run.smithi037:> hostname
2023-12-14T20:25:23.135 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com
2023-12-14T20:25:23.135 DEBUG:teuthology.orchestra.run.smithi037:> grep smithi037.front.sepia.ceph.com /etc/hosts
2023-12-14T20:25:23.183 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037
2023-12-14T20:25:23.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname smithi037
2023-12-14T20:25:23.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/smithi037.front.sepia.ceph.com/smithi037/g' /etc/hosts
2023-12-14T20:25:23.365 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/172.21.15.37/172.21.15.37/g' /etc/hosts
2023-12-14T20:25:23.419 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/smithi037.front.sepia.ceph.com/smithi037/g' /etc/hostname
2023-12-14T20:25:23.476 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname smithi037
2023-12-14T20:25:23.531 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="Ubuntu"
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="22.04"
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_CODENAME=jammy
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:ID=ubuntu
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE=debian
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-14T20:25:23.579 INFO:teuthology.orchestra.run.smithi037.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-14T20:25:23.580 INFO:teuthology.orchestra.run.smithi037.stdout:UBUNTU_CODENAME=jammy
2023-12-14T20:25:23.580 INFO:teuthology.provision.fog.smithi037:Deploy complete!
2023-12-14T20:25:23.581 DEBUG:teuthology.parallel:result is None
2023-12-14T20:25:27.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:25:27.129 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-12-14T20:25:33.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:25:33.282 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-14T20:25:33.282 DEBUG:teuthology.orchestra.run.smithi053:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-14T20:25:33.755 INFO:teuthology.provision.fog.smithi053:Node is ready
2023-12-14T20:25:33.756 DEBUG:teuthology.orchestra.run.smithi053:> hostname
2023-12-14T20:25:33.803 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053.front.sepia.ceph.com
2023-12-14T20:25:33.803 DEBUG:teuthology.orchestra.run.smithi053:> grep smithi053.front.sepia.ceph.com /etc/hosts
2023-12-14T20:25:33.855 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53 smithi053.front.sepia.ceph.com smithi053
2023-12-14T20:25:33.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053
2023-12-14T20:25:34.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hosts
2023-12-14T20:25:34.025 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/172.21.15.53/172.21.15.53/g' /etc/hosts
2023-12-14T20:25:34.083 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hostname
2023-12-14T20:25:34.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053
2023-12-14T20:25:34.198 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Ubuntu"
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="22.04"
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_CODENAME=jammy
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:ID=ubuntu
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE=debian
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-14T20:25:34.243 INFO:teuthology.orchestra.run.smithi053.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-14T20:25:34.244 INFO:teuthology.orchestra.run.smithi053.stdout:UBUNTU_CODENAME=jammy
2023-12-14T20:25:34.244 INFO:teuthology.provision.fog.smithi053:Deploy complete!
2023-12-14T20:25:34.244 DEBUG:teuthology.parallel:result is None
2023-12-14T20:25:34.429 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com
2023-12-14T20:25:34.429 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-14T20:25:34.449 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com
2023-12-14T20:25:34.449 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-14T20:25:34.467 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server
2023-12-14T20:25:34.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com'
2023-12-14T20:25:34.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:25:34.540 DEBUG:teuthology.orchestra.run.smithi037:> true
2023-12-14T20:25:34.654 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com'
2023-12-14T20:25:34.654 DEBUG:teuthology.orchestra.run.smithi037:> uname -m
2023-12-14T20:25:34.698 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64
2023-12-14T20:25:34.698 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release
2023-12-14T20:25:34.746 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="Ubuntu"
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="22.04"
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_CODENAME=jammy
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:ID=ubuntu
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE=debian
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-14T20:25:34.747 INFO:teuthology.orchestra.run.smithi037.stdout:UBUNTU_CODENAME=jammy
2023-12-14T20:25:34.748 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server
2023-12-14T20:25:34.766 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server
2023-12-14T20:25:34.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com'
2023-12-14T20:25:34.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-14T20:25:34.842 DEBUG:teuthology.orchestra.run.smithi053:> true
2023-12-14T20:25:34.958 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com'
2023-12-14T20:25:34.958 DEBUG:teuthology.orchestra.run.smithi053:> uname -m
2023-12-14T20:25:35.003 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64
2023-12-14T20:25:35.003 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release
2023-12-14T20:25:35.050 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Ubuntu"
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="22.04"
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_CODENAME=jammy
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:ID=ubuntu
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE=debian
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-14T20:25:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:UBUNTU_CODENAME=jammy
2023-12-14T20:25:35.051 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server
2023-12-14T20:25:35.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:25:35.160 INFO:teuthology.dispatcher.supervisor:Running job 7491461
2023-12-14T20:25:35.160 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2023-12-14_20:15:48-rgw-wip-cbodley-testing-distro-default-smithi/7491461 --name cbodley-2023-12-14_20:15:48-rgw-wip-cbodley-testing-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi} -- /home/teuthworker/archive/cbodley-2023-12-14_20:15:48-rgw-wip-cbodley-testing-distro-default-smithi/7491461/orig.config.yaml
2023-12-14T20:25:35.163 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2023-12-14_20:15:48-rgw-wip-cbodley-testing-distro-default-smithi/7491461
2023-12-14T20:25:35.163 INFO:teuthology.dispatcher.supervisor:Job PID: 1503109
2023-12-14T20:25:35.164 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-14T20:27:35.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:29:35.238 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:31:35.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:33:35.387 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:35:35.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:37:35.549 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:39:35.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:41:35.675 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:43:35.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:45:35.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:47:35.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:49:35.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:51:36.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:53:36.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:55:36.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:57:36.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T20:59:36.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:01:36.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:03:36.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:05:36.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:07:36.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:09:36.604 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:11:36.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:13:36.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:15:36.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:17:36.867 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:19:36.930 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:21:36.995 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:23:37.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:25:37.141 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:27:37.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:29:37.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:31:37.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:33:37.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:35:37.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:37:37.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:39:37.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:41:37.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:43:37.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:45:37.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:47:37.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:49:37.960 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:51:38.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:53:38.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:55:38.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:57:38.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T21:59:38.296 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:01:38.373 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:03:38.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:05:38.521 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:07:38.593 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:09:38.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:11:38.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:13:38.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:15:38.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:17:39.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:19:39.123 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:21:39.185 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:23:39.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:25:39.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:27:39.374 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:29:39.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:31:39.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:33:39.624 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:35:39.696 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:37:39.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:39:39.826 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:41:39.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:43:39.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:45:40.029 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:47:40.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:49:40.171 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:51:40.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:53:40.305 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:55:40.372 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:57:40.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T22:59:40.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:01:40.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:03:40.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:05:40.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:07:40.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:09:41.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:11:41.112 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:13:41.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:15:41.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:17:41.351 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:19:41.424 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:21:41.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-14T23:21:41.556 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-12-14T23:21:41.735 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-14T23:21:41.736 INFO:teuthology.nuke:Checking targets against current locks
2023-12-14T23:21:41.788 DEBUG:teuthology.nuke:shortname: smithi037
2023-12-14T23:21:41.789 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-14T23:21:41.810 DEBUG:teuthology.nuke:shortname: smithi053
2023-12-14T23:21:41.811 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-14T23:21:41.814 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-12-14_20:15:48-rgw-wip-cbodley-testing-distro-default-smithi/7491461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 20:18:52.857647', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCjEQTDyn4/gpo9FXcTeZYsUCEYkQ6WwK2OLIDz0q5Etk06wg7huoWFlw5SQuXfNhBrBvXqBfiDy9+CYAOScQs4='}
2023-12-14T23:21:41.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-12-14_20:15:48-rgw-wip-cbodley-testing-distro-default-smithi/7491461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 20:18:52.855677', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF8Dsp4RSX0yDeGYZ21esSm0RnCKEjLf2qFOyNxrPLOfo50dwt76WTKzfC798F/BLBkjTelVL8acZEp/4nfrnRc='}
2023-12-14T23:21:41.862 INFO:teuthology.orchestra.console.smithi037:Power off
2023-12-14T23:21:41.862 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-14T23:21:41.883 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off
2023-12-14T23:21:41.883 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T23:21:41.899 INFO:teuthology.orchestra.console.smithi053:Power off
2023-12-14T23:21:41.899 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-14T23:21:41.909 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off
2023-12-14T23:21:41.920 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off
2023-12-14T23:21:41.920 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-14T23:21:41.944 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off
2023-12-14T23:21:42.010 INFO:teuthology.orchestra.console.smithi037:Power off completed
2023-12-14T23:21:42.045 INFO:teuthology.orchestra.console.smithi053:Power off completed
2023-12-14T23:21:42.162 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com
2023-12-14T23:21:42.186 INFO:teuthology.lock.ops:unlocked: smithi053.front.sepia.ceph.com