2024-04-12T18:47:12.280 INFO:root:teuthology version: 0.0.1.dev267+g6c63784
2024-04-12T18:47:12.280 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T18:47:12.475 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f8b0dc7cf40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f8b0ecddc10>, signals=[15])
2024-04-12T18:47:12.476 INFO:teuthology.lock.ops:Start node 'smithi069.front.sepia.ceph.com' reimaging
2024-04-12T18:47:12.476 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: reset os type and version on server
2024-04-12T18:47:12.476 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server
2024-04-12T18:47:12.496 INFO:teuthology.lock.ops:Start node 'smithi099.front.sepia.ceph.com' reimaging
2024-04-12T18:47:12.496 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: reset os type and version on server
2024-04-12T18:47:12.496 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2024-04-12T18:47:12.521 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging
2024-04-12T18:47:12.521 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server
2024-04-12T18:47:12.522 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2024-04-12T18:47:12.622 DEBUG:teuthology.provision.fog.smithi069:Requesting image smithi_centos_9.stream (ID 10)
2024-04-12T18:47:12.644 DEBUG:teuthology.provision.fog.smithi099:Requesting image smithi_centos_9.stream (ID 10)
2024-04-12T18:47:12.670 INFO:teuthology.provision.fog.smithi069:Scheduling deploy of centos 9.stream
2024-04-12T18:47:12.674 DEBUG:teuthology.provision.fog.smithi167:Requesting image smithi_centos_9.stream (ID 10)
2024-04-12T18:47:12.689 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of centos 9.stream
2024-04-12T18:47:12.726 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 9.stream
2024-04-12T18:47:12.897 INFO:teuthology.orchestra.console.smithi069:Power off
2024-04-12T18:47:12.898 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-12T18:47:12.924 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off
2024-04-12T18:47:12.924 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:12.946 INFO:teuthology.orchestra.console.smithi099:Power off
2024-04-12T18:47:12.947 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-12T18:47:12.964 INFO:teuthology.orchestra.console.smithi167:Power off
2024-04-12T18:47:12.966 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-12T18:47:12.978 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on
2024-04-12T18:47:12.978 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off
2024-04-12T18:47:12.979 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:12.993 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off
2024-04-12T18:47:12.994 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:13.005 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-04-12T18:47:13.018 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2024-04-12T18:47:16.982 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:17.641 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:17.648 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:17.744 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on
2024-04-12T18:47:17.750 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-04-12T18:47:17.757 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2024-04-12T18:47:21.749 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:21.757 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:21.764 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:21.859 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off
2024-04-12T18:47:21.865 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2024-04-12T18:47:21.874 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2024-04-12T18:47:21.961 INFO:teuthology.orchestra.console.smithi069:Power off completed
2024-04-12T18:47:21.966 INFO:teuthology.orchestra.console.smithi099:Power off completed
2024-04-12T18:47:21.974 INFO:teuthology.orchestra.console.smithi167:Power off completed
2024-04-12T18:47:22.062 INFO:teuthology.orchestra.console.smithi069:Power on
2024-04-12T18:47:22.062 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-12T18:47:22.070 INFO:teuthology.orchestra.console.smithi099:Power on
2024-04-12T18:47:22.071 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-12T18:47:22.078 INFO:teuthology.orchestra.console.smithi167:Power on
2024-04-12T18:47:22.079 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-12T18:47:22.093 DEBUG:teuthology.orchestra.console.smithi069:power on output: Chassis Power Control: Up/On
2024-04-12T18:47:22.094 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:22.104 DEBUG:teuthology.orchestra.console.smithi099:power on output: Chassis Power Control: Up/On
2024-04-12T18:47:22.104 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:22.111 DEBUG:teuthology.orchestra.console.smithi167:power on output: Chassis Power Control: Up/On
2024-04-12T18:47:22.112 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:22.121 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off
2024-04-12T18:47:22.124 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2024-04-12T18:47:22.134 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2024-04-12T18:47:26.127 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:26.134 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:26.142 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:26.238 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on
2024-04-12T18:47:26.243 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2024-04-12T18:47:26.250 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2024-04-12T18:47:26.339 INFO:teuthology.orchestra.console.smithi069:Power on completed
2024-04-12T18:47:26.440 INFO:teuthology.provision.fog.smithi069:Waiting for deploy to finish
2024-04-12T18:47:30.247 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:30.254 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-12T18:47:30.357 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-04-12T18:47:30.364 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2024-04-12T18:47:30.458 INFO:teuthology.orchestra.console.smithi099:Power on completed
2024-04-12T18:47:30.465 INFO:teuthology.orchestra.console.smithi167:Power on completed
2024-04-12T18:47:30.559 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish
2024-04-12T18:47:30.566 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish
2024-04-12T18:50:01.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:04.482 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-04-12T18:50:10.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:12.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:13.570 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-04-12T18:50:15.518 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2024-04-12T18:50:16.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:19.550 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-12T18:50:19.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:21.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:22.626 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-04-12T18:50:24.578 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2024-04-12T18:50:25.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:28.606 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-12T18:50:28.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:30.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:31.682 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-04-12T18:50:33.666 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2024-04-12T18:50:34.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:37.662 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-12T18:50:37.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:39.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:40.734 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-04-12T18:50:42.750 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2024-04-12T18:50:43.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:46.718 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-12T18:50:46.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:48.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:49.790 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-04-12T18:50:51.806 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2024-04-12T18:50:52.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:55.774 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-12T18:50:55.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:57.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:50:58.846 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-04-12T18:51:00.866 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2024-04-12T18:51:01.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:51:04.834 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-12T18:51:04.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:51:06.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:51:10.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:52:04.905 WARNING:teuthology.provision.fog.smithi099:timed out
2024-04-12T18:52:06.873 WARNING:teuthology.provision.fog.smithi069:timed out
2024-04-12T18:52:10.846 WARNING:teuthology.provision.fog.smithi167:timed out
2024-04-12T18:52:10.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:52:11.191 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-12T18:52:11.191 DEBUG:teuthology.orchestra.run.smithi099:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-12T18:52:11.501 INFO:teuthology.provision.fog.smithi099:Node is ready
2024-04-12T18:52:11.502 DEBUG:teuthology.orchestra.run.smithi099:> hostname
2024-04-12T18:52:11.517 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com
2024-04-12T18:52:11.517 DEBUG:teuthology.orchestra.run.smithi099:> grep smithi099.front.sepia.ceph.com /etc/hosts
2024-04-12T18:52:11.572 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099
2024-04-12T18:52:11.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099
2024-04-12T18:52:11.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hosts
2024-04-12T18:52:11.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/172.21.15.99/172.21.15.99/g' /etc/hosts
2024-04-12T18:52:11.774 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hostname
2024-04-12T18:52:11.841 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099
2024-04-12T18:52:11.908 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release
2024-04-12T18:52:11.962 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream"
2024-04-12T18:52:11.962 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9"
2024-04-12T18:52:11.962 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-12T18:52:11.963 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-12T18:52:11.964 INFO:teuthology.provision.fog.smithi099:Deploy complete!
2024-04-12T18:52:11.965 DEBUG:teuthology.parallel:result is None
2024-04-12T18:52:12.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:52:13.115 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-12T18:52:13.115 DEBUG:teuthology.orchestra.run.smithi069:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-12T18:52:13.434 INFO:teuthology.provision.fog.smithi069:Node is ready
2024-04-12T18:52:13.434 DEBUG:teuthology.orchestra.run.smithi069:> hostname
2024-04-12T18:52:13.449 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069.front.sepia.ceph.com
2024-04-12T18:52:13.449 DEBUG:teuthology.orchestra.run.smithi069:> grep smithi069.front.sepia.ceph.com /etc/hosts
2024-04-12T18:52:13.504 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69 smithi069.front.sepia.ceph.com smithi069
2024-04-12T18:52:13.504 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname smithi069
2024-04-12T18:52:13.610 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi069.front.sepia.ceph.com/smithi069/g' /etc/hosts
2024-04-12T18:52:13.638 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/172.21.15.69/172.21.15.69/g' /etc/hosts
2024-04-12T18:52:13.705 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi069.front.sepia.ceph.com/smithi069/g' /etc/hostname
2024-04-12T18:52:13.772 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname smithi069
2024-04-12T18:52:13.838 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release
2024-04-12T18:52:13.892 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream"
2024-04-12T18:52:13.892 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/"
2024-04-12T18:52:13.893 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-12T18:52:13.894 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-12T18:52:13.894 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-12T18:52:13.894 INFO:teuthology.provision.fog.smithi069:Deploy complete!
2024-04-12T18:52:13.895 DEBUG:teuthology.parallel:result is None
2024-04-12T18:52:16.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:52:17.090 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-12T18:52:17.090 DEBUG:teuthology.orchestra.run.smithi167:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-12T18:52:17.416 INFO:teuthology.provision.fog.smithi167:Node is ready
2024-04-12T18:52:17.416 DEBUG:teuthology.orchestra.run.smithi167:> hostname
2024-04-12T18:52:17.431 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com
2024-04-12T18:52:17.431 DEBUG:teuthology.orchestra.run.smithi167:> grep smithi167.front.sepia.ceph.com /etc/hosts
2024-04-12T18:52:17.486 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167
2024-04-12T18:52:17.487 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167
2024-04-12T18:52:17.607 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hosts
2024-04-12T18:52:17.636 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/172.21.15.167/172.21.15.167/g' /etc/hosts
2024-04-12T18:52:17.705 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hostname
2024-04-12T18:52:17.773 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167
2024-04-12T18:52:17.840 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release
2024-04-12T18:52:17.895 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream"
2024-04-12T18:52:17.895 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9"
2024-04-12T18:52:17.895 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos"
2024-04-12T18:52:17.895 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-12T18:52:17.896 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-12T18:52:17.897 INFO:teuthology.provision.fog.smithi167:Deploy complete!
2024-04-12T18:52:17.897 DEBUG:teuthology.parallel:result is None
2024-04-12T18:52:18.115 INFO:teuthology.lock.ops:Checking smithi069.front.sepia.ceph.com
2024-04-12T18:52:18.116 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-12T18:52:18.157 INFO:teuthology.lock.ops:Checking smithi099.front.sepia.ceph.com
2024-04-12T18:52:18.157 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-12T18:52:18.182 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com
2024-04-12T18:52:18.182 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-12T18:52:18.208 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: set os type and version on server
2024-04-12T18:52:18.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com'
2024-04-12T18:52:18.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:52:18.298 DEBUG:teuthology.orchestra.run.smithi069:> true
2024-04-12T18:52:18.366 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com'
2024-04-12T18:52:18.366 DEBUG:teuthology.orchestra.run.smithi069:> uname -m
2024-04-12T18:52:18.422 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64
2024-04-12T18:52:18.422 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon"
2024-04-12T18:52:18.478 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-12T18:52:18.479 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/"
2024-04-12T18:52:18.479 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-12T18:52:18.479 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-12T18:52:18.479 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-12T18:52:18.479 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server
2024-04-12T18:52:18.499 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: set os type and version on server
2024-04-12T18:52:18.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com'
2024-04-12T18:52:18.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:52:18.594 DEBUG:teuthology.orchestra.run.smithi099:> true
2024-04-12T18:52:18.662 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com'
2024-04-12T18:52:18.662 DEBUG:teuthology.orchestra.run.smithi099:> uname -m
2024-04-12T18:52:18.719 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64
2024-04-12T18:52:18.720 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release
2024-04-12T18:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream"
2024-04-12T18:52:18.776 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-12T18:52:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-12T18:52:18.778 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2024-04-12T18:52:18.799 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server
2024-04-12T18:52:18.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2024-04-12T18:52:18.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-12T18:52:18.887 DEBUG:teuthology.orchestra.run.smithi167:> true
2024-04-12T18:52:18.911 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com'
2024-04-12T18:52:18.911 DEBUG:teuthology.orchestra.run.smithi167:> uname -m
2024-04-12T18:52:18.966 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64
2024-04-12T18:52:18.966 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/"
2024-04-12T18:52:19.021 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-12T18:52:19.022 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-12T18:52:19.022 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-12T18:52:19.022 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2024-04-12T18:52:19.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T18:52:19.181 INFO:teuthology.dispatcher.supervisor:Running job 7653482
2024-04-12T18:52:19.181 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653482 --name vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/pjd}} -- /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653482/orig.config.yaml
2024-04-12T18:52:19.184 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653482
2024-04-12T18:52:19.184 INFO:teuthology.dispatcher.supervisor:Job PID: 4130927
2024-04-12T18:52:19.184 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-04-12T18:54:19.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T18:56:19.468 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T18:58:19.649 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:00:19.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:02:19.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:04:20.157 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:06:20.322 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:08:20.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:10:20.675 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:12:20.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:14:21.012 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:16:21.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:18:21.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:20:21.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:22:21.723 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:24:21.872 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:26:22.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:28:22.215 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:30:22.383 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:32:22.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:34:22.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:36:22.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T19:36:22.974 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-04-12T19:36:23.227 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi069 but it was locked by another job: /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653540
2024-04-12T19:36:23.227 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi099 but it was locked by another job: /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653541
2024-04-12T19:36:23.227 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi167 but it was locked by another job: /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653540