2024-01-22T11:29:12.147 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-01-22T11:29:12.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:29:12.275 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f7d13434f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f7d1449ac10>, signals=[15])
2024-01-22T11:29:12.276 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging
2024-01-22T11:29:12.276 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server
2024-01-22T11:29:12.276 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2024-01-22T11:29:12.294 INFO:teuthology.lock.ops:Start node 'smithi124.front.sepia.ceph.com' reimaging
2024-01-22T11:29:12.295 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: reset os type and version on server
2024-01-22T11:29:12.295 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server
2024-01-22T11:29:12.431 DEBUG:teuthology.provision.fog.smithi027:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-01-22T11:29:12.440 DEBUG:teuthology.provision.fog.smithi124:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-01-22T11:29:12.480 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of ubuntu 22.04
2024-01-22T11:29:12.483 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of ubuntu 22.04
2024-01-22T11:29:12.706 INFO:teuthology.orchestra.console.smithi124:Power off
2024-01-22T11:29:12.707 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-22T11:29:12.720 INFO:teuthology.orchestra.console.smithi027:Power off
2024-01-22T11:29:12.720 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-22T11:29:12.730 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off
2024-01-22T11:29:12.730 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:12.741 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off
2024-01-22T11:29:12.742 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:12.751 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on
2024-01-22T11:29:12.763 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on
2024-01-22T11:29:16.756 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:16.764 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:16.864 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on
2024-01-22T11:29:16.873 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on
2024-01-22T11:29:20.868 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:20.875 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:20.977 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on
2024-01-22T11:29:20.983 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on
2024-01-22T11:29:24.982 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:24.990 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:25.093 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off
2024-01-22T11:29:25.098 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2024-01-22T11:29:25.194 INFO:teuthology.orchestra.console.smithi124:Power off completed
2024-01-22T11:29:25.199 INFO:teuthology.orchestra.console.smithi027:Power off completed
2024-01-22T11:29:25.295 INFO:teuthology.orchestra.console.smithi124:Power on
2024-01-22T11:29:25.295 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-22T11:29:25.303 INFO:teuthology.orchestra.console.smithi027:Power on
2024-01-22T11:29:25.304 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-22T11:29:25.316 DEBUG:teuthology.orchestra.console.smithi124:power on output: Chassis Power Control: Up/On
2024-01-22T11:29:25.317 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:25.327 DEBUG:teuthology.orchestra.console.smithi027:power on output: Chassis Power Control: Up/On
2024-01-22T11:29:25.327 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:25.343 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off
2024-01-22T11:29:25.350 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2024-01-22T11:29:29.348 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:29.356 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:29.458 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on
2024-01-22T11:29:29.464 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2024-01-22T11:29:29.560 INFO:teuthology.orchestra.console.smithi124:Power on completed
2024-01-22T11:29:29.660 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish
2024-01-22T11:29:33.468 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T11:29:33.577 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on
2024-01-22T11:29:33.678 INFO:teuthology.orchestra.console.smithi027:Power on completed
2024-01-22T11:29:33.779 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish
2024-01-22T11:32:45.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:32:48.928 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124
2024-01-22T11:32:54.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:32:58.012 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124
2024-01-22T11:33:04.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:05.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:07.104 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124
2024-01-22T11:33:08.124 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:33:13.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:14.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:16.188 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124
2024-01-22T11:33:17.180 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:33:22.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:23.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:25.244 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124
2024-01-22T11:33:26.240 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:33:31.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:32.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:34.300 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124
2024-01-22T11:33:35.292 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:33:40.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:41.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:43.388 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124
2024-01-22T11:33:44.348 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:33:49.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:50.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:33:53.404 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:33:59.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:34:02.460 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:34:08.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:34:11.516 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:34:17.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:34:20.572 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-01-22T11:34:26.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:34:49.416 WARNING:teuthology.provision.fog.smithi124:timed out
2024-01-22T11:34:55.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:34:55.649 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-22T11:34:55.650 DEBUG:teuthology.orchestra.run.smithi124:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-22T11:34:56.076 INFO:teuthology.provision.fog.smithi124:Node is ready
2024-01-22T11:34:56.076 DEBUG:teuthology.orchestra.run.smithi124:> hostname
2024-01-22T11:34:56.123 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com
2024-01-22T11:34:56.123 DEBUG:teuthology.orchestra.run.smithi124:> grep smithi124.front.sepia.ceph.com /etc/hosts
2024-01-22T11:34:56.172 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124
2024-01-22T11:34:56.172 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124
2024-01-22T11:34:56.308 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi124.front.sepia.ceph.com/smithi124/g' /etc/hosts
2024-01-22T11:34:56.321 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/172.21.15.124/172.21.15.124/g' /etc/hosts
2024-01-22T11:34:56.374 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi124.front.sepia.ceph.com/smithi124/g' /etc/hostname
2024-01-22T11:34:56.431 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124
2024-01-22T11:34:56.486 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release
2024-01-22T11:34:56.535 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-01-22T11:34:56.535 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Ubuntu"
2024-01-22T11:34:56.535 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="22.04"
2024-01-22T11:34:56.535 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-01-22T11:34:56.535 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_CODENAME=jammy
2024-01-22T11:34:56.535 INFO:teuthology.orchestra.run.smithi124.stdout:ID=ubuntu
2024-01-22T11:34:56.535 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE=debian
2024-01-22T11:34:56.536 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-22T11:34:56.536 INFO:teuthology.orchestra.run.smithi124.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-22T11:34:56.536 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-22T11:34:56.536 INFO:teuthology.orchestra.run.smithi124.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-22T11:34:56.536 INFO:teuthology.orchestra.run.smithi124.stdout:UBUNTU_CODENAME=jammy
2024-01-22T11:34:56.536 INFO:teuthology.provision.fog.smithi124:Deploy complete!
2024-01-22T11:34:56.538 DEBUG:teuthology.parallel:result is None
2024-01-22T11:35:26.608 WARNING:teuthology.provision.fog.smithi027:timed out
2024-01-22T11:35:32.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:35:32.781 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-22T11:35:32.781 DEBUG:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-22T11:35:33.223 INFO:teuthology.provision.fog.smithi027:Node is ready
2024-01-22T11:35:33.224 DEBUG:teuthology.orchestra.run.smithi027:> hostname
2024-01-22T11:35:33.272 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com
2024-01-22T11:35:33.273 DEBUG:teuthology.orchestra.run.smithi027:> grep smithi027.front.sepia.ceph.com /etc/hosts
2024-01-22T11:35:33.320 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027
2024-01-22T11:35:33.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027
2024-01-22T11:35:33.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hosts
2024-01-22T11:35:33.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/172.21.15.27/172.21.15.27/g' /etc/hosts
2024-01-22T11:35:33.557 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hostname
2024-01-22T11:35:33.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027
2024-01-22T11:35:33.671 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release
2024-01-22T11:35:33.720 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-01-22T11:35:33.720 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="Ubuntu"
2024-01-22T11:35:33.720 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="22.04"
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_CODENAME=jammy
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:ID=ubuntu
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE=debian
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-22T11:35:33.721 INFO:teuthology.orchestra.run.smithi027.stdout:UBUNTU_CODENAME=jammy
2024-01-22T11:35:33.721 INFO:teuthology.provision.fog.smithi027:Deploy complete!
2024-01-22T11:35:33.722 DEBUG:teuthology.parallel:result is None
2024-01-22T11:35:33.942 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com
2024-01-22T11:35:33.942 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-22T11:35:33.962 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com
2024-01-22T11:35:33.962 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-22T11:35:33.980 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server
2024-01-22T11:35:33.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com'
2024-01-22T11:35:33.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:35:34.067 DEBUG:teuthology.orchestra.run.smithi027:> true
2024-01-22T11:35:34.180 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com'
2024-01-22T11:35:34.181 DEBUG:teuthology.orchestra.run.smithi027:> uname -m
2024-01-22T11:35:34.228 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64
2024-01-22T11:35:34.228 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release
2024-01-22T11:35:34.275 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-01-22T11:35:34.276 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="Ubuntu"
2024-01-22T11:35:34.276 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="22.04"
2024-01-22T11:35:34.276 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-01-22T11:35:34.276 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_CODENAME=jammy
2024-01-22T11:35:34.276 INFO:teuthology.orchestra.run.smithi027.stdout:ID=ubuntu
2024-01-22T11:35:34.276 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE=debian
2024-01-22T11:35:34.277 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-22T11:35:34.277 INFO:teuthology.orchestra.run.smithi027.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-22T11:35:34.277 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-22T11:35:34.277 INFO:teuthology.orchestra.run.smithi027.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-22T11:35:34.277 INFO:teuthology.orchestra.run.smithi027.stdout:UBUNTU_CODENAME=jammy
2024-01-22T11:35:34.277 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2024-01-22T11:35:34.296 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server
2024-01-22T11:35:34.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com'
2024-01-22T11:35:34.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-22T11:35:34.386 DEBUG:teuthology.orchestra.run.smithi124:> true
2024-01-22T11:35:34.636 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com'
2024-01-22T11:35:34.636 DEBUG:teuthology.orchestra.run.smithi124:> uname -m
2024-01-22T11:35:34.682 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64
2024-01-22T11:35:34.683 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release
2024-01-22T11:35:34.730 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-01-22T11:35:34.730 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Ubuntu"
2024-01-22T11:35:34.730 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="22.04"
2024-01-22T11:35:34.730 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-01-22T11:35:34.731 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_CODENAME=jammy
2024-01-22T11:35:34.731 INFO:teuthology.orchestra.run.smithi124.stdout:ID=ubuntu
2024-01-22T11:35:34.731 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE=debian
2024-01-22T11:35:34.731 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-22T11:35:34.731 INFO:teuthology.orchestra.run.smithi124.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-22T11:35:34.731 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-22T11:35:34.731 INFO:teuthology.orchestra.run.smithi124.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-22T11:35:34.731 INFO:teuthology.orchestra.run.smithi124.stdout:UBUNTU_CODENAME=jammy
2024-01-22T11:35:34.731 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server
2024-01-22T11:35:34.754 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:35:34.840 INFO:teuthology.dispatcher.supervisor:Running job 7523724
2024-01-22T11:35:34.840 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/bin/teuthology -v --owner scheduled_suriarte@teuthology --archive /home/teuthworker/archive/suriarte-2024-01-19_18:38:49-rgw-rgw_asio_frontend_asserts-distro-default-smithi/7523724 --name suriarte-2024-01-19_18:38:49-rgw-rgw_asio_frontend_asserts-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi} -- /home/teuthworker/archive/suriarte-2024-01-19_18:38:49-rgw-rgw_asio_frontend_asserts-distro-default-smithi/7523724/orig.config.yaml
2024-01-22T11:35:34.843 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/suriarte-2024-01-19_18:38:49-rgw-rgw_asio_frontend_asserts-distro-default-smithi/7523724
2024-01-22T11:35:34.844 INFO:teuthology.dispatcher.supervisor:Job PID: 1010613
2024-01-22T11:35:34.844 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-22T11:37:34.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:39:35.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:41:35.284 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:43:35.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:45:35.648 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:47:35.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:49:36.004 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:51:36.109 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:53:36.272 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:55:36.428 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:57:36.592 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T11:59:36.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:01:37.149 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:03:37.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:05:37.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:07:37.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:09:37.786 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:11:37.957 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:13:38.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:15:38.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:17:38.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:19:38.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:21:38.797 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:23:38.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:25:39.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:27:39.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:29:39.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:31:39.584 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:33:39.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:35:39.903 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:37:40.064 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:39:40.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:41:40.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:43:40.524 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:45:40.689 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:47:40.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:49:41.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:51:41.188 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:53:41.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:55:41.496 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:57:41.597 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T12:59:41.756 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:01:41.921 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:03:42.064 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:05:42.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:07:42.328 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:09:42.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:11:42.648 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:13:42.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:15:42.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:17:43.145 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:19:43.317 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:21:43.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:23:43.600 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:25:43.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:27:43.954 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:29:44.128 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:31:44.309 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:33:44.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:35:44.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:37:44.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:39:44.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:41:45.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:43:45.290 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:45:45.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:47:45.597 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:49:45.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:51:45.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:53:46.089 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:55:46.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:57:46.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T13:59:46.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:01:46.808 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:03:46.985 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:05:47.149 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:07:47.324 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:09:47.485 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:11:47.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:13:47.800 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:15:47.924 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:17:48.092 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:19:48.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:21:48.328 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:23:48.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:25:48.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:27:48.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:29:48.996 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:31:49.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:33:49.324 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:35:49.496 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:37:49.665 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:39:49.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-22T14:39:49.911 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-22T14:39:50.155 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-22T14:39:50.155 INFO:teuthology.nuke:Checking targets against current locks
2024-01-22T14:39:50.201 DEBUG:teuthology.nuke:shortname: smithi027
2024-01-22T14:39:50.201 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-22T14:39:50.226 DEBUG:teuthology.nuke:shortname: smithi124
2024-01-22T14:39:50.226 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-22T14:39:50.230 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-01-19_18:38:49-rgw-rgw_asio_frontend_asserts-distro-default-smithi/7523724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-22 11:29:11.135372', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPjCYLF+1vbZ2XcmZp25MXa1wmJTfCatoKq59n95NSjnWll2CFamOuTl4TbwDNrAPXjNtAITOqHA9eMu/YU8RYA='}
2024-01-22T14:39:50.249 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-01-19_18:38:49-rgw-rgw_asio_frontend_asserts-distro-default-smithi/7523724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-22 11:29:11.134039', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAdf8CFdacvAqJhr1Z1G4jHk6Kv26wc8Y5k1SDJIFf1X991hgJNfBgaePKPsmpByJ7521IhihXzjmK9eU1rSRTQ='}
2024-01-22T14:39:50.276 INFO:teuthology.orchestra.console.smithi027:Power off
2024-01-22T14:39:50.276 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-22T14:39:50.298 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off
2024-01-22T14:39:50.299 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T14:39:50.318 INFO:teuthology.orchestra.console.smithi124:Power off
2024-01-22T14:39:50.318 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-22T14:39:50.326 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2024-01-22T14:39:50.340 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off
2024-01-22T14:39:50.340 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-22T14:39:50.360 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off
2024-01-22T14:39:50.427 INFO:teuthology.orchestra.console.smithi027:Power off completed
2024-01-22T14:39:50.462 INFO:teuthology.orchestra.console.smithi124:Power off completed
2024-01-22T14:39:50.574 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com
2024-01-22T14:39:50.609 INFO:teuthology.lock.ops:unlocked: smithi124.front.sepia.ceph.com