2024-03-16T08:06:13.206 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-16T08:06:13.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:06:13.377 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f463b56df70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f463c5cec10>, signals=[15])
2024-03-16T08:06:13.378 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging
2024-03-16T08:06:13.378 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server
2024-03-16T08:06:13.378 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2024-03-16T08:06:13.397 INFO:teuthology.lock.ops:Start node 'smithi172.front.sepia.ceph.com' reimaging
2024-03-16T08:06:13.397 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: reset os type and version on server
2024-03-16T08:06:13.397 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server
2024-03-16T08:06:13.422 INFO:teuthology.lock.ops:Start node 'smithi203.front.sepia.ceph.com' reimaging
2024-03-16T08:06:13.422 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: reset os type and version on server
2024-03-16T08:06:13.422 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2024-03-16T08:06:13.523 DEBUG:teuthology.provision.fog.smithi169:Requesting image smithi_centos_9.stream (ID 10)
2024-03-16T08:06:13.531 DEBUG:teuthology.provision.fog.smithi172:Requesting image smithi_centos_9.stream (ID 10)
2024-03-16T08:06:13.564 DEBUG:teuthology.provision.fog.smithi203:Requesting image smithi_centos_9.stream (ID 10)
2024-03-16T08:06:13.571 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of centos 9.stream
2024-03-16T08:06:13.586 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 9.stream
2024-03-16T08:06:13.620 INFO:teuthology.provision.fog.smithi203:Scheduling deploy of centos 9.stream
2024-03-16T08:06:13.647 DEBUG:teuthology.provision.fog.smithi172:Canceling deploy task with ID 864214
2024-03-16T08:06:13.948 INFO:teuthology.orchestra.console.smithi169:Power off
2024-03-16T08:06:13.948 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-16T08:06:13.962 INFO:teuthology.orchestra.console.smithi203:Power off
2024-03-16T08:06:13.962 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-16T08:06:13.972 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off
2024-03-16T08:06:13.973 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:13.987 INFO:teuthology.orchestra.console.smithi172:Power off
2024-03-16T08:06:13.987 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-16T08:06:13.994 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off
2024-03-16T08:06:13.995 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:14.001 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on
2024-03-16T08:06:14.008 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off
2024-03-16T08:06:14.008 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:14.015 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on
2024-03-16T08:06:14.028 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off
2024-03-16T08:06:14.128 INFO:teuthology.orchestra.console.smithi172:Power off completed
2024-03-16T08:06:14.229 INFO:teuthology.orchestra.console.smithi172:Power on
2024-03-16T08:06:14.229 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-16T08:06:14.250 DEBUG:teuthology.orchestra.console.smithi172:power on output: Chassis Power Control: Up/On
2024-03-16T08:06:14.250 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:14.270 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off
2024-03-16T08:06:18.005 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:18.016 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:18.114 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on
2024-03-16T08:06:18.124 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on
2024-03-16T08:06:18.271 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:18.379 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off
2024-03-16T08:06:22.118 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:22.125 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:22.228 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on
2024-03-16T08:06:22.234 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2024-03-16T08:06:22.335 INFO:teuthology.orchestra.console.smithi203:Power off completed
2024-03-16T08:06:22.380 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:22.435 INFO:teuthology.orchestra.console.smithi203:Power on
2024-03-16T08:06:22.436 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-16T08:06:22.458 DEBUG:teuthology.orchestra.console.smithi203:power on output: Chassis Power Control: Up/On
2024-03-16T08:06:22.458 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:22.478 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2024-03-16T08:06:22.489 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on
2024-03-16T08:06:22.590 INFO:teuthology.orchestra.console.smithi172:Power on completed
2024-03-16T08:06:22.691 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish
2024-03-16T08:06:26.232 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:26.341 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2024-03-16T08:06:26.442 INFO:teuthology.orchestra.console.smithi169:Power off completed
2024-03-16T08:06:26.479 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:26.543 INFO:teuthology.orchestra.console.smithi169:Power on
2024-03-16T08:06:26.543 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-16T08:06:26.563 DEBUG:teuthology.orchestra.console.smithi169:power on output: Chassis Power Control: Up/On
2024-03-16T08:06:26.563 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:26.583 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2024-03-16T08:06:26.588 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2024-03-16T08:06:30.588 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:30.595 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-16T08:06:30.697 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on
2024-03-16T08:06:30.703 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on
2024-03-16T08:06:30.799 INFO:teuthology.orchestra.console.smithi169:Power on completed
2024-03-16T08:06:30.804 INFO:teuthology.orchestra.console.smithi203:Power on completed
2024-03-16T08:06:30.900 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish
2024-03-16T08:06:30.905 INFO:teuthology.provision.fog.smithi203:Waiting for deploy to finish
2024-03-16T08:09:08.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:16.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:16.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:26.819 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172
2024-03-16T08:09:32.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:35.266 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:09:35.267 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-03-16T08:09:35.902 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172
2024-03-16T08:09:41.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:41.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:41.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:44.350 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:09:44.351 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-03-16T08:09:44.958 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172
2024-03-16T08:09:50.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:50.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:50.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:53.438 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:09:53.439 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-03-16T08:09:54.014 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172
2024-03-16T08:09:59.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:09:59.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:00.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:02.530 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:10:02.531 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-03-16T08:10:03.074 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172
2024-03-16T08:10:08.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:08.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:09.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:11.614 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:10:11.615 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-03-16T08:10:17.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:17.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:20.670 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:10:20.702 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-03-16T08:10:26.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:26.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:45.155 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:10:51.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:10:54.238 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:11:00.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:11:03.326 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2024-03-16T08:11:09.084 WARNING:teuthology.provision.fog.smithi172:timed out
2024-03-16T08:11:09.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:11:09.557 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-16T08:11:09.558 DEBUG:teuthology.orchestra.run.smithi169:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-16T08:11:15.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:11:15.372 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-16T08:11:15.372 DEBUG:teuthology.orchestra.run.smithi172:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-16T08:11:15.693 INFO:teuthology.provision.fog.smithi172:Node is ready
2024-03-16T08:11:15.694 DEBUG:teuthology.orchestra.run.smithi172:> hostname
2024-03-16T08:11:15.708 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com
2024-03-16T08:11:15.708 DEBUG:teuthology.orchestra.run.smithi172:> grep smithi172.front.sepia.ceph.com /etc/hosts
2024-03-16T08:11:15.763 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172
2024-03-16T08:11:15.764 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172
2024-03-16T08:11:15.874 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi172.front.sepia.ceph.com/smithi172/g' /etc/hosts
2024-03-16T08:11:15.902 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/172.21.15.172/172.21.15.172/g' /etc/hosts
2024-03-16T08:11:15.969 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi172.front.sepia.ceph.com/smithi172/g' /etc/hostname
2024-03-16T08:11:16.036 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172
2024-03-16T08:11:16.102 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release
2024-03-16T08:11:16.156 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream"
2024-03-16T08:11:16.156 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9"
2024-03-16T08:11:16.156 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos"
2024-03-16T08:11:16.156 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora"
2024-03-16T08:11:16.156 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9"
2024-03-16T08:11:16.156 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9"
2024-03-16T08:11:16.156 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-16T08:11:16.157 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31"
2024-03-16T08:11:16.157 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon"
2024-03-16T08:11:16.157 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-16T08:11:16.157 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/"
2024-03-16T08:11:16.157 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-16T08:11:16.157 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-16T08:11:16.157 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-16T08:11:16.158 INFO:teuthology.provision.fog.smithi172:Deploy complete!
2024-03-16T08:11:16.159 DEBUG:teuthology.parallel:result is None
2024-03-16T08:11:26.704 WARNING:teuthology.provision.fog.smithi203:timed out
2024-03-16T08:11:29.884 INFO:teuthology.provision.fog.smithi169:Node is ready
2024-03-16T08:11:29.884 DEBUG:teuthology.orchestra.run.smithi169:> hostname
2024-03-16T08:11:29.902 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com
2024-03-16T08:11:29.903 DEBUG:teuthology.orchestra.run.smithi169:> grep smithi169.front.sepia.ceph.com /etc/hosts
2024-03-16T08:11:29.960 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169
2024-03-16T08:11:29.961 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169
2024-03-16T08:11:30.089 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hosts
2024-03-16T08:11:30.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/172.21.15.169/172.21.15.169/g' /etc/hosts
2024-03-16T08:11:30.189 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hostname
2024-03-16T08:11:30.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169
2024-03-16T08:11:30.327 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release
2024-03-16T08:11:30.382 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-16T08:11:30.383 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-16T08:11:30.384 INFO:teuthology.provision.fog.smithi169:Deploy complete!
2024-03-16T08:11:30.384 DEBUG:teuthology.parallel:result is None
2024-03-16T08:11:32.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:11:33.030 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-16T08:11:33.030 DEBUG:teuthology.orchestra.run.smithi203:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-16T08:11:33.376 INFO:teuthology.provision.fog.smithi203:Node is ready
2024-03-16T08:11:33.376 DEBUG:teuthology.orchestra.run.smithi203:> hostname
2024-03-16T08:11:33.391 INFO:teuthology.orchestra.run.smithi203.stdout:smithi203.front.sepia.ceph.com
2024-03-16T08:11:33.391 DEBUG:teuthology.orchestra.run.smithi203:> grep smithi203.front.sepia.ceph.com /etc/hosts
2024-03-16T08:11:33.447 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.15.203 smithi203.front.sepia.ceph.com smithi203
2024-03-16T08:11:33.448 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname smithi203
2024-03-16T08:11:33.543 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/smithi203.front.sepia.ceph.com/smithi203/g' /etc/hosts
2024-03-16T08:11:33.572 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/172.21.15.203/172.21.15.203/g' /etc/hosts
2024-03-16T08:11:33.640 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/smithi203.front.sepia.ceph.com/smithi203/g' /etc/hostname
2024-03-16T08:11:33.709 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname smithi203
2024-03-16T08:11:33.775 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release
2024-03-16T08:11:33.830 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-16T08:11:33.831 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-16T08:11:33.832 INFO:teuthology.provision.fog.smithi203:Deploy complete!
2024-03-16T08:11:33.832 DEBUG:teuthology.parallel:result is None
2024-03-16T08:11:34.041 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com
2024-03-16T08:11:34.042 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-16T08:11:34.067 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com
2024-03-16T08:11:34.067 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-16T08:11:34.083 INFO:teuthology.lock.ops:Checking smithi203.front.sepia.ceph.com
2024-03-16T08:11:34.083 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-16T08:11:34.101 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server
2024-03-16T08:11:34.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2024-03-16T08:11:34.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:11:34.189 DEBUG:teuthology.orchestra.run.smithi169:> true
2024-03-16T08:11:34.262 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com'
2024-03-16T08:11:34.262 DEBUG:teuthology.orchestra.run.smithi169:> uname -m
2024-03-16T08:11:34.318 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64
2024-03-16T08:11:34.319 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/"
2024-03-16T08:11:34.375 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-16T08:11:34.376 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-16T08:11:34.376 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-16T08:11:34.376 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2024-03-16T08:11:34.400 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server
2024-03-16T08:11:34.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com'
2024-03-16T08:11:34.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:11:34.485 DEBUG:teuthology.orchestra.run.smithi172:> true
2024-03-16T08:11:34.657 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com'
2024-03-16T08:11:34.657 DEBUG:teuthology.orchestra.run.smithi172:> uname -m
2024-03-16T08:11:34.712 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64
2024-03-16T08:11:34.712 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-16T08:11:34.767 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-16T08:11:34.768 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-16T08:11:34.768 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server
2024-03-16T08:11:34.809 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: set os type and version on server
2024-03-16T08:11:34.809 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com'
2024-03-16T08:11:34.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-16T08:11:34.893 DEBUG:teuthology.orchestra.run.smithi203:> true
2024-03-16T08:11:34.966 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com'
2024-03-16T08:11:34.966 DEBUG:teuthology.orchestra.run.smithi203:> uname -m
2024-03-16T08:11:35.020 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64
2024-03-16T08:11:35.021 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-16T08:11:35.076 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-16T08:11:35.077 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2024-03-16T08:11:35.100 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:11:35.205 INFO:teuthology.dispatcher.supervisor:Running job 7602289
2024-03-16T08:11:35.206 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602289 --name pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/pjd}} -- /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602289/orig.config.yaml
2024-03-16T08:11:35.208 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602289
2024-03-16T08:11:35.209 INFO:teuthology.dispatcher.supervisor:Job PID: 1165938
2024-03-16T08:11:35.209 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-16T08:13:35.307 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:15:35.483 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:17:35.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:19:35.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:21:35.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:23:36.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:25:36.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:27:36.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:29:36.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:31:36.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:33:36.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:35:37.077 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:37:37.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:39:37.407 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:41:37.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:43:37.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:45:37.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:47:38.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:49:38.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:51:38.437 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-16T08:51:38.506 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1