2024-02-18T10:20:53.474 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55
2024-02-18T10:20:53.474 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:20:53.657 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f2cc3cecf40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f2cc4d60c10>, signals=[15])
2024-02-18T10:20:53.658 INFO:teuthology.lock.ops:Start node 'smithi089.front.sepia.ceph.com' reimaging
2024-02-18T10:20:53.658 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: reset os type and version on server
2024-02-18T10:20:53.658 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2024-02-18T10:20:53.678 INFO:teuthology.lock.ops:Start node 'smithi117.front.sepia.ceph.com' reimaging
2024-02-18T10:20:53.678 INFO:teuthology.lock.ops:Updating [smithi117.front.sepia.ceph.com]: reset os type and version on server
2024-02-18T10:20:53.678 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server
2024-02-18T10:20:53.704 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging
2024-02-18T10:20:53.704 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server
2024-02-18T10:20:53.704 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2024-02-18T10:20:53.795 DEBUG:teuthology.provision.fog.smithi089:Requesting image smithi_centos_9.stream (ID 10)
2024-02-18T10:20:53.811 DEBUG:teuthology.provision.fog.smithi117:Requesting image smithi_centos_9.stream (ID 10)
2024-02-18T10:20:53.847 DEBUG:teuthology.provision.fog.smithi167:Requesting image smithi_centos_9.stream (ID 10)
2024-02-18T10:20:53.850 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of centos 9.stream
2024-02-18T10:20:53.853 INFO:teuthology.provision.fog.smithi117:Scheduling deploy of centos 9.stream
2024-02-18T10:20:53.886 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 9.stream
2024-02-18T10:20:54.174 INFO:teuthology.orchestra.console.smithi117:Power off
2024-02-18T10:20:54.174 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-18T10:20:54.189 INFO:teuthology.orchestra.console.smithi167:Power off
2024-02-18T10:20:54.189 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-18T10:20:54.200 INFO:teuthology.orchestra.console.smithi089:Power off
2024-02-18T10:20:54.200 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-18T10:20:54.208 DEBUG:teuthology.orchestra.console.smithi117:power off output: Chassis Power Control: Down/Off
2024-02-18T10:20:54.208 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:54.217 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off
2024-02-18T10:20:54.217 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:54.225 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off
2024-02-18T10:20:54.226 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:54.234 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on
2024-02-18T10:20:54.241 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2024-02-18T10:20:54.245 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off
2024-02-18T10:20:54.342 INFO:teuthology.orchestra.console.smithi167:Power off completed
2024-02-18T10:20:54.346 INFO:teuthology.orchestra.console.smithi089:Power off completed
2024-02-18T10:20:54.443 INFO:teuthology.orchestra.console.smithi167:Power on
2024-02-18T10:20:54.443 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-18T10:20:54.453 INFO:teuthology.orchestra.console.smithi089:Power on
2024-02-18T10:20:54.454 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-18T10:20:54.471 DEBUG:teuthology.orchestra.console.smithi167:power on output: Chassis Power Control: Up/On
2024-02-18T10:20:54.471 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:54.480 DEBUG:teuthology.orchestra.console.smithi089:power on output: Chassis Power Control: Up/On
2024-02-18T10:20:54.480 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:54.493 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2024-02-18T10:20:54.499 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off
2024-02-18T10:20:58.236 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:58.344 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off
2024-02-18T10:20:58.445 INFO:teuthology.orchestra.console.smithi117:Power off completed
2024-02-18T10:20:58.494 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:58.501 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:58.546 INFO:teuthology.orchestra.console.smithi117:Power on
2024-02-18T10:20:58.547 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-18T10:20:58.568 DEBUG:teuthology.orchestra.console.smithi117:power on output: Chassis Power Control: Up/On
2024-02-18T10:20:58.569 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:20:58.589 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off
2024-02-18T10:20:58.603 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2024-02-18T10:20:58.608 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on
2024-02-18T10:20:58.709 INFO:teuthology.orchestra.console.smithi089:Power on completed
2024-02-18T10:20:58.809 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish
2024-02-18T10:21:02.594 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:21:02.604 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-18T10:21:02.704 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on
2024-02-18T10:21:02.712 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2024-02-18T10:21:02.805 INFO:teuthology.orchestra.console.smithi117:Power on completed
2024-02-18T10:21:02.813 INFO:teuthology.orchestra.console.smithi167:Power on completed
2024-02-18T10:21:02.906 INFO:teuthology.provision.fog.smithi117:Waiting for deploy to finish
2024-02-18T10:21:02.914 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish
2024-02-18T10:23:30.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:23:30.340 ERROR:teuthology.orchestra.connection:Error authenticating with smithi089.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-02-18T10:23:34.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:23:37.212 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:23:43.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:23:46.272 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:23:49.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:23:52.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:23:52.316 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-02-18T10:23:55.324 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:23:58.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:01.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:01.376 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-02-18T10:24:04.380 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:24:07.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:10.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:10.432 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-02-18T10:24:13.436 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:24:16.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:19.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:19.484 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-02-18T10:24:22.492 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:24:25.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:28.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:28.540 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-02-18T10:24:31.548 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:24:34.343 WARNING:teuthology.provision.fog.smithi089:timed out
2024-02-18T10:24:34.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:37.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:37.600 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-02-18T10:24:40.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:43.424 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-02-18T10:24:43.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:46.652 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-02-18T10:24:49.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:52.512 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-02-18T10:24:52.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:24:55.712 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-02-18T10:24:55.868 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:24:58.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:25:01.564 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-02-18T10:25:01.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:25:01.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:25:04.924 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:25:07.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:25:10.624 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-02-18T10:25:10.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:25:13.984 WARNING:teuthology.provision.fog.smithi117:[Errno None] Unable to connect to port 22 on 172.21.15.117
2024-02-18T10:25:16.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:25:16.881 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-18T10:25:16.882 DEBUG:teuthology.orchestra.run.smithi089:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-18T10:25:19.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:25:20.237 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-18T10:25:20.237 DEBUG:teuthology.orchestra.run.smithi117:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-18T10:25:35.662 INFO:teuthology.provision.fog.smithi117:Node is ready
2024-02-18T10:25:35.662 DEBUG:teuthology.orchestra.run.smithi117:> hostname
2024-02-18T10:25:35.681 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117.front.sepia.ceph.com
2024-02-18T10:25:35.681 DEBUG:teuthology.orchestra.run.smithi117:> grep smithi117.front.sepia.ceph.com /etc/hosts
2024-02-18T10:25:35.739 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117 smithi117.front.sepia.ceph.com smithi117
2024-02-18T10:25:35.739 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname smithi117
2024-02-18T10:25:35.845 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i -e 's/smithi117.front.sepia.ceph.com/smithi117/g' /etc/hosts
2024-02-18T10:25:35.875 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i -e 's/172.21.15.117/172.21.15.117/g' /etc/hosts
2024-02-18T10:25:35.944 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i -e 's/smithi117.front.sepia.ceph.com/smithi117/g' /etc/hostname
2024-02-18T10:25:36.013 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname smithi117
2024-02-18T10:25:36.079 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release
2024-02-18T10:25:36.134 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream"
2024-02-18T10:25:36.134 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9"
2024-02-18T10:25:36.134 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos"
2024-02-18T10:25:36.134 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora"
2024-02-18T10:25:36.134 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-18T10:25:36.135 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-18T10:25:36.136 INFO:teuthology.provision.fog.smithi117:Deploy complete!
2024-02-18T10:25:36.137 DEBUG:teuthology.parallel:result is None
2024-02-18T10:25:37.210 INFO:teuthology.provision.fog.smithi089:Node is ready
2024-02-18T10:25:37.211 DEBUG:teuthology.orchestra.run.smithi089:> hostname
2024-02-18T10:25:37.229 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com
2024-02-18T10:25:37.229 DEBUG:teuthology.orchestra.run.smithi089:> grep smithi089.front.sepia.ceph.com /etc/hosts
2024-02-18T10:25:37.287 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089
2024-02-18T10:25:37.287 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089
2024-02-18T10:25:37.410 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hosts
2024-02-18T10:25:37.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/172.21.15.89/172.21.15.89/g' /etc/hosts
2024-02-18T10:25:37.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hostname
2024-02-18T10:25:37.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089
2024-02-18T10:25:37.650 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon"
2024-02-18T10:25:37.706 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-18T10:25:37.707 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/"
2024-02-18T10:25:37.707 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-18T10:25:37.707 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-18T10:25:37.707 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-18T10:25:37.707 INFO:teuthology.provision.fog.smithi089:Deploy complete!
2024-02-18T10:25:37.708 DEBUG:teuthology.parallel:result is None
2024-02-18T10:26:01.717 WARNING:teuthology.provision.fog.smithi167:timed out
2024-02-18T10:26:07.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:26:07.999 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-18T10:26:07.999 DEBUG:teuthology.orchestra.run.smithi167:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-18T10:26:08.342 INFO:teuthology.provision.fog.smithi167:Node is ready
2024-02-18T10:26:08.343 DEBUG:teuthology.orchestra.run.smithi167:> hostname
2024-02-18T10:26:08.357 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com
2024-02-18T10:26:08.357 DEBUG:teuthology.orchestra.run.smithi167:> grep smithi167.front.sepia.ceph.com /etc/hosts
2024-02-18T10:26:08.412 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167
2024-02-18T10:26:08.413 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167
2024-02-18T10:26:08.524 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hosts
2024-02-18T10:26:08.551 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/172.21.15.167/172.21.15.167/g' /etc/hosts
2024-02-18T10:26:08.618 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hostname
2024-02-18T10:26:08.685 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167
2024-02-18T10:26:08.752 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release
2024-02-18T10:26:08.806 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/"
2024-02-18T10:26:08.807 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-18T10:26:08.808 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-18T10:26:08.808 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-18T10:26:08.808 INFO:teuthology.provision.fog.smithi167:Deploy complete!
2024-02-18T10:26:08.809 DEBUG:teuthology.parallel:result is None
2024-02-18T10:26:09.019 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com
2024-02-18T10:26:09.019 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-18T10:26:09.061 INFO:teuthology.lock.ops:Checking smithi117.front.sepia.ceph.com
2024-02-18T10:26:09.061 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-18T10:26:09.085 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com
2024-02-18T10:26:09.086 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-18T10:26:09.104 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server
2024-02-18T10:26:09.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com'
2024-02-18T10:26:09.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:26:09.216 DEBUG:teuthology.orchestra.run.smithi089:> true
2024-02-18T10:26:09.390 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com'
2024-02-18T10:26:09.390 DEBUG:teuthology.orchestra.run.smithi089:> uname -m
2024-02-18T10:26:09.445 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64
2024-02-18T10:26:09.445 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release
2024-02-18T10:26:09.499 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream"
2024-02-18T10:26:09.499 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9"
2024-02-18T10:26:09.499 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos"
2024-02-18T10:26:09.499 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-18T10:26:09.500 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-18T10:26:09.501 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2024-02-18T10:26:09.528 INFO:teuthology.lock.ops:Updating [smithi117.front.sepia.ceph.com]: set os type and version on server
2024-02-18T10:26:09.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com'
2024-02-18T10:26:09.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:26:09.618 DEBUG:teuthology.orchestra.run.smithi117:> true
2024-02-18T10:26:09.787 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com'
2024-02-18T10:26:09.787 DEBUG:teuthology.orchestra.run.smithi117:> uname -m
2024-02-18T10:26:09.841 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64
2024-02-18T10:26:09.842 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release
2024-02-18T10:26:09.896 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-18T10:26:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-18T10:26:09.898 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server
2024-02-18T10:26:09.919 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server
2024-02-18T10:26:09.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2024-02-18T10:26:09.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-18T10:26:10.005 DEBUG:teuthology.orchestra.run.smithi167:> true
2024-02-18T10:26:10.074 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com'
2024-02-18T10:26:10.075 DEBUG:teuthology.orchestra.run.smithi167:> uname -m
2024-02-18T10:26:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64
2024-02-18T10:26:10.129 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon"
2024-02-18T10:26:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-18T10:26:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/"
2024-02-18T10:26:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-18T10:26:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-18T10:26:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-18T10:26:10.184 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2024-02-18T10:26:10.202 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:26:10.305 INFO:teuthology.dispatcher.supervisor:Running job 7564287
2024-02-18T10:26:10.305 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564287 --name vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/direct_io}} -- /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564287/orig.config.yaml
2024-02-18T10:26:10.310 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564287
2024-02-18T10:26:10.310 INFO:teuthology.dispatcher.supervisor:Job PID: 1934416
2024-02-18T10:26:10.310 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-02-18T10:28:10.312 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:30:10.395 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:32:10.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:34:10.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:36:10.701 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:38:10.776 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:40:10.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:42:10.956 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:44:11.018 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:46:11.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:48:11.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:50:11.353 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:52:11.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-18T10:52:11.546 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-02-18T10:52:11.786 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi089 but it was locked by another job: /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564331
2024-02-18T10:52:11.787 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi117 but it was locked by another job: /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564330
2024-02-18T10:52:11.787 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi167 but it was locked by another job: /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564330