2023-12-19T08:39:26.771 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-19T08:39:26.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T08:39:27.001 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f6d8f7355b0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f6d9068b940>, signals=[15])
2023-12-19T08:39:27.002 INFO:teuthology.lock.ops:Start node 'smithi038.front.sepia.ceph.com' reimaging
2023-12-19T08:39:27.002 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: reset os type and version on server
2023-12-19T08:39:27.002 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2023-12-19T08:39:27.027 INFO:teuthology.lock.ops:Start node 'smithi110.front.sepia.ceph.com' reimaging
2023-12-19T08:39:27.027 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: reset os type and version on server
2023-12-19T08:39:27.027 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2023-12-19T08:39:27.052 INFO:teuthology.lock.ops:Start node 'smithi160.front.sepia.ceph.com' reimaging
2023-12-19T08:39:27.052 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: reset os type and version on server
2023-12-19T08:39:27.052 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server
2023-12-19T08:39:27.144 DEBUG:teuthology.provision.fog.smithi038:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-12-19T08:39:27.168 DEBUG:teuthology.provision.fog.smithi110:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-12-19T08:39:27.192 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of ubuntu 20.04
2023-12-19T08:39:27.195 DEBUG:teuthology.provision.fog.smithi160:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-12-19T08:39:27.218 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of ubuntu 20.04
2023-12-19T08:39:27.240 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of ubuntu 20.04
2023-12-19T08:39:27.445 INFO:teuthology.orchestra.console.smithi038:Power off
2023-12-19T08:39:27.445 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T08:39:27.459 INFO:teuthology.orchestra.console.smithi160:Power off
2023-12-19T08:39:27.460 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T08:39:27.471 INFO:teuthology.orchestra.console.smithi110:Power off
2023-12-19T08:39:27.471 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T08:39:27.480 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off
2023-12-19T08:39:27.480 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:27.488 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off
2023-12-19T08:39:27.488 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:27.498 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off
2023-12-19T08:39:27.498 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:27.508 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2023-12-19T08:39:27.512 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off
2023-12-19T08:39:27.520 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2023-12-19T08:39:27.609 INFO:teuthology.orchestra.console.smithi038:Power off completed
2023-12-19T08:39:27.613 INFO:teuthology.orchestra.console.smithi160:Power off completed
2023-12-19T08:39:27.710 INFO:teuthology.orchestra.console.smithi038:Power on
2023-12-19T08:39:27.710 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-19T08:39:27.718 INFO:teuthology.orchestra.console.smithi160:Power on
2023-12-19T08:39:27.718 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-19T08:39:27.733 DEBUG:teuthology.orchestra.console.smithi038:power on output: Chassis Power Control: Up/On
2023-12-19T08:39:27.734 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:27.742 DEBUG:teuthology.orchestra.console.smithi160:power on output: Chassis Power Control: Up/On
2023-12-19T08:39:27.742 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:27.754 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2023-12-19T08:39:27.761 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off
2023-12-19T08:39:31.521 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:31.633 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2023-12-19T08:39:31.755 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:31.765 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:31.867 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2023-12-19T08:39:31.874 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on
2023-12-19T08:39:31.974 INFO:teuthology.orchestra.console.smithi160:Power on completed
2023-12-19T08:39:32.075 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish
2023-12-19T08:39:35.637 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:35.746 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2023-12-19T08:39:35.847 INFO:teuthology.orchestra.console.smithi110:Power off completed
2023-12-19T08:39:35.867 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:35.948 INFO:teuthology.orchestra.console.smithi110:Power on
2023-12-19T08:39:35.949 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-19T08:39:35.971 DEBUG:teuthology.orchestra.console.smithi110:power on output: Chassis Power Control: Up/On
2023-12-19T08:39:35.971 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:35.979 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on
2023-12-19T08:39:35.992 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2023-12-19T08:39:36.080 INFO:teuthology.orchestra.console.smithi038:Power on completed
2023-12-19T08:39:36.181 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2023-12-19T08:39:39.993 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:40.102 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2023-12-19T08:39:44.105 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T08:39:44.317 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2023-12-19T08:39:44.418 INFO:teuthology.orchestra.console.smithi110:Power on completed
2023-12-19T08:39:44.519 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish
2023-12-19T08:43:07.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:43:07.377 ERROR:teuthology.orchestra.connection:Error authenticating with smithi038.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-19T08:43:15.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:43:15.805 ERROR:teuthology.orchestra.connection:Error authenticating with smithi110.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-19T08:43:18.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:43:21.329 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2023-12-19T08:43:27.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:43:30.385 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2023-12-19T08:43:36.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:43:39.446 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2023-12-19T08:43:45.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:43:48.497 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2023-12-19T08:43:54.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:43:57.553 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2023-12-19T08:44:03.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:06.609 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2023-12-19T08:44:11.380 WARNING:teuthology.provision.fog.smithi038:timed out
2023-12-19T08:44:12.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:15.665 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2023-12-19T08:44:17.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:19.808 WARNING:teuthology.provision.fog.smithi110:timed out
2023-12-19T08:44:21.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:25.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:28.885 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2023-12-19T08:44:34.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:35.729 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-19T08:44:37.937 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2023-12-19T08:44:41.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:43.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:44.785 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-19T08:44:47.029 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2023-12-19T08:44:50.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:53.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:44:53.379 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-19T08:44:53.379 DEBUG:teuthology.orchestra.run.smithi110:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-19T08:44:53.842 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-19T08:44:59.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:45:00.061 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-19T08:45:00.061 DEBUG:teuthology.orchestra.run.smithi038:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-19T08:45:00.534 INFO:teuthology.provision.fog.smithi038:Node is ready
2023-12-19T08:45:00.534 DEBUG:teuthology.orchestra.run.smithi038:> hostname
2023-12-19T08:45:00.540 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2023-12-19T08:45:00.541 DEBUG:teuthology.orchestra.run.smithi038:> grep smithi038.front.sepia.ceph.com /etc/hosts
2023-12-19T08:45:00.590 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2023-12-19T08:45:00.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038
2023-12-19T08:45:00.679 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hosts
2023-12-19T08:45:00.696 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/172.21.15.38/172.21.15.38/g' /etc/hosts
2023-12-19T08:45:00.756 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hostname
2023-12-19T08:45:00.769 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038
2023-12-19T08:45:00.781 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release
2023-12-19T08:45:00.786 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Ubuntu"
2023-12-19T08:45:00.786 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-12-19T08:45:00.786 INFO:teuthology.orchestra.run.smithi038.stdout:ID=ubuntu
2023-12-19T08:45:00.786 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE=debian
2023-12-19T08:45:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-12-19T08:45:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="20.04"
2023-12-19T08:45:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-19T08:45:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-19T08:45:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-19T08:45:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-19T08:45:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_CODENAME=focal
2023-12-19T08:45:00.787 INFO:teuthology.orchestra.run.smithi038.stdout:UBUNTU_CODENAME=focal
2023-12-19T08:45:00.788 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2023-12-19T08:45:00.789 DEBUG:teuthology.parallel:result is None
2023-12-19T08:45:14.501 INFO:teuthology.provision.fog.smithi110:Node is ready
2023-12-19T08:45:14.501 DEBUG:teuthology.orchestra.run.smithi110:> hostname
2023-12-19T08:45:14.508 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com
2023-12-19T08:45:14.508 DEBUG:teuthology.orchestra.run.smithi110:> grep smithi110.front.sepia.ceph.com /etc/hosts
2023-12-19T08:45:14.515 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110
2023-12-19T08:45:14.515 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110
2023-12-19T08:45:14.583 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hosts
2023-12-19T08:45:14.599 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/172.21.15.110/172.21.15.110/g' /etc/hosts
2023-12-19T08:45:14.658 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hostname
2023-12-19T08:45:14.714 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110
2023-12-19T08:45:14.773 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release
2023-12-19T08:45:14.778 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Ubuntu"
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:ID=ubuntu
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE=debian
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="20.04"
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_CODENAME=focal
2023-12-19T08:45:14.779 INFO:teuthology.orchestra.run.smithi110.stdout:UBUNTU_CODENAME=focal
2023-12-19T08:45:14.780 INFO:teuthology.provision.fog.smithi110:Deploy complete!
2023-12-19T08:45:14.781 DEBUG:teuthology.parallel:result is None
2023-12-19T08:45:21.669 WARNING:teuthology.provision.fog.smithi160:timed out
2023-12-19T08:45:27.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:45:27.807 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-19T08:45:27.807 DEBUG:teuthology.orchestra.run.smithi160:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-19T08:45:28.277 INFO:teuthology.provision.fog.smithi160:Node is ready
2023-12-19T08:45:28.277 DEBUG:teuthology.orchestra.run.smithi160:> hostname
2023-12-19T08:45:28.283 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com
2023-12-19T08:45:28.283 DEBUG:teuthology.orchestra.run.smithi160:> grep smithi160.front.sepia.ceph.com /etc/hosts
2023-12-19T08:45:28.291 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160
2023-12-19T08:45:28.291 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160
2023-12-19T08:45:28.467 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hosts
2023-12-19T08:45:28.484 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/172.21.15.160/172.21.15.160/g' /etc/hosts
2023-12-19T08:45:28.496 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hostname
2023-12-19T08:45:28.555 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160
2023-12-19T08:45:28.565 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release
2023-12-19T08:45:28.571 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Ubuntu"
2023-12-19T08:45:28.572 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-12-19T08:45:28.572 INFO:teuthology.orchestra.run.smithi160.stdout:ID=ubuntu
2023-12-19T08:45:28.572 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE=debian
2023-12-19T08:45:28.572 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-12-19T08:45:28.572 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="20.04"
2023-12-19T08:45:28.572 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-19T08:45:28.572 INFO:teuthology.orchestra.run.smithi160.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-19T08:45:28.573 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-19T08:45:28.573 INFO:teuthology.orchestra.run.smithi160.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-19T08:45:28.573 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_CODENAME=focal
2023-12-19T08:45:28.573 INFO:teuthology.orchestra.run.smithi160.stdout:UBUNTU_CODENAME=focal
2023-12-19T08:45:28.573 INFO:teuthology.provision.fog.smithi160:Deploy complete!
2023-12-19T08:45:28.574 DEBUG:teuthology.parallel:result is None
2023-12-19T08:45:28.899 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com
2023-12-19T08:45:28.900 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com
2023-12-19T08:45:28.900 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com
2023-12-19T08:45:28.901 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server
2023-12-19T08:45:28.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2023-12-19T08:45:28.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:45:28.953 DEBUG:teuthology.orchestra.run.smithi038:> true
2023-12-19T08:45:29.083 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com'
2023-12-19T08:45:29.083 DEBUG:teuthology.orchestra.run.smithi038:> uname -m
2023-12-19T08:45:29.132 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64
2023-12-19T08:45:29.132 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release
2023-12-19T08:45:29.139 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Ubuntu"
2023-12-19T08:45:29.139 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-12-19T08:45:29.139 INFO:teuthology.orchestra.run.smithi038.stdout:ID=ubuntu
2023-12-19T08:45:29.139 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE=debian
2023-12-19T08:45:29.139 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-12-19T08:45:29.139 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="20.04"
2023-12-19T08:45:29.139 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-19T08:45:29.140 INFO:teuthology.orchestra.run.smithi038.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-19T08:45:29.140 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-19T08:45:29.140 INFO:teuthology.orchestra.run.smithi038.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-19T08:45:29.140 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_CODENAME=focal
2023-12-19T08:45:29.140 INFO:teuthology.orchestra.run.smithi038.stdout:UBUNTU_CODENAME=focal
2023-12-19T08:45:29.140 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2023-12-19T08:45:29.168 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server
2023-12-19T08:45:29.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com'
2023-12-19T08:45:29.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:45:29.222 DEBUG:teuthology.orchestra.run.smithi110:> true
2023-12-19T08:45:29.423 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com'
2023-12-19T08:45:29.424 DEBUG:teuthology.orchestra.run.smithi110:> uname -m
2023-12-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64
2023-12-19T08:45:29.430 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release
2023-12-19T08:45:29.437 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Ubuntu"
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:ID=ubuntu
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE=debian
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="20.04"
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_CODENAME=focal
2023-12-19T08:45:29.438 INFO:teuthology.orchestra.run.smithi110.stdout:UBUNTU_CODENAME=focal
2023-12-19T08:45:29.439 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2023-12-19T08:45:29.460 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server
2023-12-19T08:45:29.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com'
2023-12-19T08:45:29.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T08:45:29.513 DEBUG:teuthology.orchestra.run.smithi160:> true
2023-12-19T08:45:29.627 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com'
2023-12-19T08:45:29.628 DEBUG:teuthology.orchestra.run.smithi160:> uname -m
2023-12-19T08:45:29.634 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64
2023-12-19T08:45:29.634 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release
2023-12-19T08:45:29.639 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Ubuntu"
2023-12-19T08:45:29.639 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-12-19T08:45:29.639 INFO:teuthology.orchestra.run.smithi160.stdout:ID=ubuntu
2023-12-19T08:45:29.639 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE=debian
2023-12-19T08:45:29.639 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-12-19T08:45:29.639 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="20.04"
2023-12-19T08:45:29.640 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-19T08:45:29.640 INFO:teuthology.orchestra.run.smithi160.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-19T08:45:29.640 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-19T08:45:29.640 INFO:teuthology.orchestra.run.smithi160.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-19T08:45:29.640 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_CODENAME=focal
2023-12-19T08:45:29.640 INFO:teuthology.orchestra.run.smithi160.stdout:UBUNTU_CODENAME=focal
2023-12-19T08:45:29.640 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server
2023-12-19T08:45:29.669 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T08:45:29.777 INFO:teuthology.dispatcher.supervisor:Running job 7495994
2023-12-19T08:45:29.777 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-12-19_05:14:21-fs-rishabh-mdscap-pacific-testing-default-smithi/7495994 --name rishabh-2023-12-19_05:14:21-fs-rishabh-mdscap-pacific-testing-default-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} -- /home/teuthworker/archive/rishabh-2023-12-19_05:14:21-fs-rishabh-mdscap-pacific-testing-default-smithi/7495994/orig.config.yaml
2023-12-19T08:45:29.780 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-12-19_05:14:21-fs-rishabh-mdscap-pacific-testing-default-smithi/7495994
2023-12-19T08:45:29.781 INFO:teuthology.dispatcher.supervisor:Job PID: 965877
2023-12-19T08:45:29.781 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-19T08:47:29.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T08:49:29.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T08:51:29.949 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T08:53:30.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T08:55:30.107 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T08:57:30.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T08:59:30.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T09:01:30.352 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T09:03:30.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T09:05:30.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T09:05:30.606 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-12-19T09:05:30.794 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-19T09:05:30.795 INFO:teuthology.nuke:Checking targets against current locks
2023-12-19T09:05:30.850 DEBUG:teuthology.nuke:shortname: smithi038
2023-12-19T09:05:30.851 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-19T09:05:30.879 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_05:14:21-fs-rishabh-mdscap-pacific-testing-default-smithi/7495994', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 08:39:25.875049', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}
2023-12-19T09:05:30.880 DEBUG:teuthology.nuke:shortname: smithi110
2023-12-19T09:05:30.880 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-19T09:05:30.901 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_05:14:21-fs-rishabh-mdscap-pacific-testing-default-smithi/7495994', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 08:39:25.876384', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}
2023-12-19T09:05:30.907 DEBUG:teuthology.nuke:shortname: smithi160
2023-12-19T09:05:30.907 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-19T09:05:30.930 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_05:14:21-fs-rishabh-mdscap-pacific-testing-default-smithi/7495994', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 08:39:25.873418', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}
2023-12-19T09:05:30.944 INFO:teuthology.orchestra.console.smithi038:Power off
2023-12-19T09:05:30.944 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T09:05:30.967 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off
2023-12-19T09:05:30.967 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T09:05:30.989 INFO:teuthology.orchestra.console.smithi110:Power off
2023-12-19T09:05:30.989 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T09:05:31.003 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2023-12-19T09:05:31.013 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off
2023-12-19T09:05:31.013 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T09:05:31.031 INFO:teuthology.orchestra.console.smithi160:Power off
2023-12-19T09:05:31.032 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T09:05:31.049 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2023-12-19T09:05:31.064 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off
2023-12-19T09:05:31.064 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T09:05:31.089 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off
2023-12-19T09:05:31.104 INFO:teuthology.orchestra.console.smithi038:Power off completed
2023-12-19T09:05:31.150 INFO:teuthology.orchestra.console.smithi110:Power off completed
2023-12-19T09:05:31.191 INFO:teuthology.orchestra.console.smithi160:Power off completed
2023-12-19T09:05:31.267 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com
2023-12-19T09:05:31.298 INFO:teuthology.lock.ops:unlocked: smithi110.front.sepia.ceph.com
2023-12-19T09:05:31.340 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com