2023-12-11T10:18:22.017 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-11T10:18:22.017 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:18:22.138 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f13c4153550>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f13c5099940>, signals=[15])
2023-12-11T10:18:22.138 INFO:teuthology.lock.ops:Start node 'smithi066.front.sepia.ceph.com' reimaging
2023-12-11T10:18:22.139 INFO:teuthology.lock.ops:Updating [smithi066.front.sepia.ceph.com]: reset os type and version on server
2023-12-11T10:18:22.139 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server
2023-12-11T10:18:22.160 INFO:teuthology.lock.ops:Start node 'smithi183.front.sepia.ceph.com' reimaging
2023-12-11T10:18:22.160 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: reset os type and version on server
2023-12-11T10:18:22.160 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2023-12-11T10:18:22.276 DEBUG:teuthology.provision.fog.smithi066:Requesting image smithi_centos_8.stream (ID 3)
2023-12-11T10:18:22.295 DEBUG:teuthology.provision.fog.smithi183:Requesting image smithi_centos_8.stream (ID 3)
2023-12-11T10:18:22.325 INFO:teuthology.provision.fog.smithi066:Scheduling deploy of centos 8.stream
2023-12-11T10:18:22.338 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of centos 8.stream
2023-12-11T10:18:22.609 INFO:teuthology.orchestra.console.smithi183:Power off
2023-12-11T10:18:22.610 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-11T10:18:22.623 INFO:teuthology.orchestra.console.smithi066:Power off
2023-12-11T10:18:22.623 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-11T10:18:22.644 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off
2023-12-11T10:18:22.644 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:22.655 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off
2023-12-11T10:18:22.656 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:22.667 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on
2023-12-11T10:18:22.676 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on
2023-12-11T10:18:26.670 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:26.678 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:26.780 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on
2023-12-11T10:18:26.786 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on
2023-12-11T10:18:30.782 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:30.789 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:30.892 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off
2023-12-11T10:18:30.898 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2023-12-11T10:18:30.993 INFO:teuthology.orchestra.console.smithi066:Power off completed
2023-12-11T10:18:30.999 INFO:teuthology.orchestra.console.smithi183:Power off completed
2023-12-11T10:18:31.094 INFO:teuthology.orchestra.console.smithi066:Power on
2023-12-11T10:18:31.094 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-11T10:18:31.101 INFO:teuthology.orchestra.console.smithi183:Power on
2023-12-11T10:18:31.101 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-11T10:18:31.113 DEBUG:teuthology.orchestra.console.smithi066:power on output: Chassis Power Control: Up/On
2023-12-11T10:18:31.113 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:31.121 DEBUG:teuthology.orchestra.console.smithi183:power on output: Chassis Power Control: Up/On
2023-12-11T10:18:31.121 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:31.134 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off
2023-12-11T10:18:31.139 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2023-12-11T10:18:35.139 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:35.145 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:35.247 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off
2023-12-11T10:18:35.253 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2023-12-11T10:18:39.252 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:39.260 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T10:18:39.363 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on
2023-12-11T10:18:39.464 INFO:teuthology.orchestra.console.smithi066:Power on completed
2023-12-11T10:18:39.468 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on
2023-12-11T10:18:39.565 INFO:teuthology.provision.fog.smithi066:Waiting for deploy to finish
2023-12-11T10:18:39.570 INFO:teuthology.orchestra.console.smithi183:Power on completed
2023-12-11T10:18:39.671 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish
2023-12-11T10:21:56.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:21:56.946 ERROR:teuthology.orchestra.connection:Error authenticating with smithi066.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-11T10:22:11.298 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:22:17.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:22:20.354 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:22:26.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:22:29.410 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:22:35.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:22:38.466 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:22:44.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:22:47.522 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:22:53.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:22:56.578 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:23:02.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:23:05.634 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:23:11.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:23:14.690 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:23:20.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:23:23.746 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:23:27.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:23:29.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:23:32.802 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T10:23:38.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:23:45.766 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-12-11T10:23:51.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:23:54.850 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-12-11T10:24:00.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:24:03.906 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-12-11T10:24:09.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:24:12.962 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-12-11T10:24:18.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:24:22.018 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-12-11T10:24:28.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:24:38.242 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-12-11T10:24:38.806 WARNING:teuthology.provision.fog.smithi066:timed out
2023-12-11T10:24:44.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:24:44.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:24:45.051 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-11T10:24:45.052 DEBUG:teuthology.orchestra.run.smithi066:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-11T10:24:45.329 INFO:teuthology.provision.fog.smithi066:Node is ready
2023-12-11T10:24:45.329 DEBUG:teuthology.orchestra.run.smithi066:> hostname
2023-12-11T10:24:45.350 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066.front.sepia.ceph.com
2023-12-11T10:24:45.350 DEBUG:teuthology.orchestra.run.smithi066:> grep smithi066.front.sepia.ceph.com /etc/hosts
2023-12-11T10:24:45.412 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66 smithi066.front.sepia.ceph.com smithi066
2023-12-11T10:24:45.412 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname smithi066
2023-12-11T10:24:45.510 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i -e 's/smithi066.front.sepia.ceph.com/smithi066/g' /etc/hosts
2023-12-11T10:24:45.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i -e 's/172.21.15.66/172.21.15.66/g' /etc/hosts
2023-12-11T10:24:45.611 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i -e 's/smithi066.front.sepia.ceph.com/smithi066/g' /etc/hostname
2023-12-11T10:24:45.681 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname smithi066
2023-12-11T10:24:45.750 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release
2023-12-11T10:24:45.809 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream"
2023-12-11T10:24:45.809 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8"
2023-12-11T10:24:45.809 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos"
2023-12-11T10:24:45.809 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora"
2023-12-11T10:24:45.809 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8"
2023-12-11T10:24:45.809 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8"
2023-12-11T10:24:45.809 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-11T10:24:45.809 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31"
2023-12-11T10:24:45.810 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-11T10:24:45.810 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/"
2023-12-11T10:24:45.810 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-11T10:24:45.810 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-11T10:24:45.810 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-11T10:24:45.810 INFO:teuthology.provision.fog.smithi066:Deploy complete!
2023-12-11T10:24:45.812 DEBUG:teuthology.parallel:result is None
2023-12-11T10:24:47.298 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-12-11T10:24:53.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:24:56.354 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-12-11T10:25:02.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:25:02.592 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-11T10:25:02.593 DEBUG:teuthology.orchestra.run.smithi183:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-11T10:25:22.766 INFO:teuthology.provision.fog.smithi183:Node is ready
2023-12-11T10:25:22.767 DEBUG:teuthology.orchestra.run.smithi183:> hostname
2023-12-11T10:25:22.790 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com
2023-12-11T10:25:22.791 DEBUG:teuthology.orchestra.run.smithi183:> grep smithi183.front.sepia.ceph.com /etc/hosts
2023-12-11T10:25:22.851 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183
2023-12-11T10:25:22.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183
2023-12-11T10:25:22.974 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hosts
2023-12-11T10:25:23.005 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/172.21.15.183/172.21.15.183/g' /etc/hosts
2023-12-11T10:25:23.074 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hostname
2023-12-11T10:25:23.143 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183
2023-12-11T10:25:23.209 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release
2023-12-11T10:25:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream"
2023-12-11T10:25:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8"
2023-12-11T10:25:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos"
2023-12-11T10:25:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora"
2023-12-11T10:25:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8"
2023-12-11T10:25:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8"
2023-12-11T10:25:23.268 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-11T10:25:23.268 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31"
2023-12-11T10:25:23.268 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-11T10:25:23.268 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/"
2023-12-11T10:25:23.268 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-11T10:25:23.268 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-11T10:25:23.268 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-11T10:25:23.268 INFO:teuthology.provision.fog.smithi183:Deploy complete!
2023-12-11T10:25:23.269 DEBUG:teuthology.parallel:result is None
2023-12-11T10:25:23.420 INFO:teuthology.lock.ops:Checking smithi066.front.sepia.ceph.com
2023-12-11T10:25:23.420 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-11T10:25:23.440 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com
2023-12-11T10:25:23.440 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-11T10:25:23.458 INFO:teuthology.lock.ops:Updating [smithi066.front.sepia.ceph.com]: set os type and version on server
2023-12-11T10:25:23.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com'
2023-12-11T10:25:23.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:25:23.527 DEBUG:teuthology.orchestra.run.smithi066:> true
2023-12-11T10:25:23.669 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com'
2023-12-11T10:25:23.669 DEBUG:teuthology.orchestra.run.smithi066:> uname -m
2023-12-11T10:25:23.728 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64
2023-12-11T10:25:23.729 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release
2023-12-11T10:25:23.787 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream"
2023-12-11T10:25:23.787 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-11T10:25:23.788 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-11T10:25:23.789 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server
2023-12-11T10:25:23.807 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server
2023-12-11T10:25:23.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com'
2023-12-11T10:25:23.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T10:25:23.872 DEBUG:teuthology.orchestra.run.smithi183:> true
2023-12-11T10:25:23.942 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com'
2023-12-11T10:25:23.942 DEBUG:teuthology.orchestra.run.smithi183:> uname -m
2023-12-11T10:25:24.000 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64
2023-12-11T10:25:24.001 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release
2023-12-11T10:25:24.059 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream"
2023-12-11T10:25:24.059 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8"
2023-12-11T10:25:24.059 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-11T10:25:24.060 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-11T10:25:24.061 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2023-12-11T10:25:24.082 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:25:24.173 INFO:teuthology.dispatcher.supervisor:Running job 7486742
2023-12-11T10:25:24.173 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner mchangir --archive /home/teuthworker/archive/mchangir-2023-12-11_10:17:09-fs:upgrade-wip-mchangir-testing-20231205.064351-testing-default-smithi/7486742 --name mchangir-2023-12-11_10:17:09-fs:upgrade-wip-mchangir-testing-20231205.064351-testing-default-smithi --description fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/mchangir-2023-12-11_10:17:09-fs:upgrade-wip-mchangir-testing-20231205.064351-testing-default-smithi/7486742/orig.config.yaml
2023-12-11T10:25:24.176 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2023-12-11_10:17:09-fs:upgrade-wip-mchangir-testing-20231205.064351-testing-default-smithi/7486742
2023-12-11T10:25:24.176 INFO:teuthology.dispatcher.supervisor:Job PID: 3343329
2023-12-11T10:25:24.177 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-11T10:27:24.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:29:24.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:31:24.328 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:33:24.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:35:24.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:37:24.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:39:24.596 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:41:24.662 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:43:24.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:45:24.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:47:24.861 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:49:24.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:51:24.993 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:53:25.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:55:25.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:57:25.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T10:59:25.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:01:25.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:03:25.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:05:25.521 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:07:25.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:09:25.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:11:25.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:13:25.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:15:25.947 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:17:26.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:19:26.090 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:21:26.156 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:23:26.226 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:25:26.476 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:27:26.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:29:26.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:31:26.689 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:33:26.754 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:35:26.825 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:37:26.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:39:26.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:41:27.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:43:27.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:45:27.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:47:27.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:49:27.393 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:51:27.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:53:27.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:55:27.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:57:27.690 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T11:59:27.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T12:01:27.826 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T12:03:27.897 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T12:03:27.963 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-12-11T12:03:28.067 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-11T12:03:28.067 INFO:teuthology.nuke:Checking targets against current locks
2023-12-11T12:03:28.113 DEBUG:teuthology.nuke:shortname: smithi066
2023-12-11T12:03:28.113 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-11T12:03:28.130 DEBUG:teuthology.nuke:shortname: smithi183
2023-12-11T12:03:28.130 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-11T12:03:28.133 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-11_10:17:09-fs:upgrade-wip-mchangir-testing-20231205.064351-testing-default-smithi/7486742', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 10:18:21.183614', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBALK874c7Ap7wh79a8oJgndEsdi/EH/oSlmuB9ZucovNQMVC3bGzshR0WhkjBAgcTHzNLs2qErZ6BGa92BRHDow='}
2023-12-11T12:03:28.151 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-11_10:17:09-fs:upgrade-wip-mchangir-testing-20231205.064351-testing-default-smithi/7486742', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 10:18:21.181931', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOCsgA55fBvMbqcz8N1/WyLwXYljSw3Pnqza7yUHzS++1m4tcclaKhp3rPhYSN3Ggt+uWBXbeV/X2JhITLVE1cg='}
2023-12-11T12:03:28.178 INFO:teuthology.orchestra.console.smithi066:Power off
2023-12-11T12:03:28.179 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-11T12:03:28.200 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off
2023-12-11T12:03:28.200 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T12:03:28.217 INFO:teuthology.orchestra.console.smithi183:Power off
2023-12-11T12:03:28.217 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-11T12:03:28.225 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off
2023-12-11T12:03:28.237 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off
2023-12-11T12:03:28.237 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T12:03:28.257 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2023-12-11T12:03:28.326 INFO:teuthology.orchestra.console.smithi066:Power off completed
2023-12-11T12:03:28.357 INFO:teuthology.orchestra.console.smithi183:Power off completed
2023-12-11T12:03:28.478 INFO:teuthology.lock.ops:unlocked: smithi066.front.sepia.ceph.com
2023-12-11T12:03:28.504 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com