2024-03-13T22:53:30.763 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-13T22:53:30.764 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T22:53:30.948 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7ff432326f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7ff433387c10>, signals=[15])
2024-03-13T22:53:30.948 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging
2024-03-13T22:53:30.948 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server
2024-03-13T22:53:30.948 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2024-03-13T22:53:30.967 INFO:teuthology.lock.ops:Start node 'smithi064.front.sepia.ceph.com' reimaging
2024-03-13T22:53:30.967 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: reset os type and version on server
2024-03-13T22:53:30.967 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server
2024-03-13T22:53:30.991 INFO:teuthology.lock.ops:Start node 'smithi163.front.sepia.ceph.com' reimaging
2024-03-13T22:53:30.992 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: reset os type and version on server
2024-03-13T22:53:30.992 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server
2024-03-13T22:53:31.086 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_centos_9.stream (ID 10)
2024-03-13T22:53:31.111 DEBUG:teuthology.provision.fog.smithi064:Requesting image smithi_centos_9.stream (ID 10)
2024-03-13T22:53:31.131 DEBUG:teuthology.provision.fog.smithi163:Requesting image smithi_centos_9.stream (ID 10)
2024-03-13T22:53:31.141 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 9.stream
2024-03-13T22:53:31.159 INFO:teuthology.provision.fog.smithi064:Scheduling deploy of centos 9.stream
2024-03-13T22:53:31.181 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of centos 9.stream
2024-03-13T22:53:31.402 INFO:teuthology.orchestra.console.smithi023:Power off
2024-03-13T22:53:31.403 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-13T22:53:31.417 INFO:teuthology.orchestra.console.smithi064:Power off
2024-03-13T22:53:31.418 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-13T22:53:31.429 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off
2024-03-13T22:53:31.429 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:31.440 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off
2024-03-13T22:53:31.440 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:31.451 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2024-03-13T22:53:31.456 INFO:teuthology.orchestra.console.smithi163:Power off
2024-03-13T22:53:31.456 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-13T22:53:31.464 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on
2024-03-13T22:53:31.476 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off
2024-03-13T22:53:31.477 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:31.499 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2024-03-13T22:53:35.454 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:35.464 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:35.500 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:35.564 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2024-03-13T22:53:35.572 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on
2024-03-13T22:53:35.608 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2024-03-13T22:53:39.566 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:39.573 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:39.608 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:39.679 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2024-03-13T22:53:39.681 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off
2024-03-13T22:53:39.717 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2024-03-13T22:53:39.780 INFO:teuthology.orchestra.console.smithi023:Power off completed
2024-03-13T22:53:39.781 INFO:teuthology.orchestra.console.smithi064:Power off completed
2024-03-13T22:53:39.881 INFO:teuthology.orchestra.console.smithi023:Power on
2024-03-13T22:53:39.881 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-13T22:53:39.888 INFO:teuthology.orchestra.console.smithi064:Power on
2024-03-13T22:53:39.889 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-13T22:53:39.905 DEBUG:teuthology.orchestra.console.smithi023:power on output: Chassis Power Control: Up/On
2024-03-13T22:53:39.905 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:39.914 DEBUG:teuthology.orchestra.console.smithi064:power on output: Chassis Power Control: Up/On
2024-03-13T22:53:39.914 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:39.927 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2024-03-13T22:53:39.936 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off
2024-03-13T22:53:43.721 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:43.830 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2024-03-13T22:53:43.928 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:43.935 INFO:teuthology.orchestra.console.smithi163:Power off completed
2024-03-13T22:53:43.937 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:44.036 INFO:teuthology.orchestra.console.smithi163:Power on
2024-03-13T22:53:44.036 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-13T22:53:44.043 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2024-03-13T22:53:44.046 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off
2024-03-13T22:53:44.061 DEBUG:teuthology.orchestra.console.smithi163:power on output: Chassis Power Control: Up/On
2024-03-13T22:53:44.061 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:44.083 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2024-03-13T22:53:48.046 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:48.058 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:48.084 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:48.160 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2024-03-13T22:53:48.168 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on
2024-03-13T22:53:48.193 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2024-03-13T22:53:48.261 INFO:teuthology.orchestra.console.smithi023:Power on completed
2024-03-13T22:53:48.269 INFO:teuthology.orchestra.console.smithi064:Power on completed
2024-03-13T22:53:48.362 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish
2024-03-13T22:53:48.370 INFO:teuthology.provision.fog.smithi064:Waiting for deploy to finish
2024-03-13T22:53:52.198 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-13T22:53:52.306 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2024-03-13T22:53:52.407 INFO:teuthology.orchestra.console.smithi163:Power on completed
2024-03-13T22:53:52.508 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish
2024-03-13T22:56:19.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:56:22.302 WARNING:teuthology.provision.fog.smithi064:[Errno None] Unable to connect to port 22 on 172.21.15.64
2024-03-13T22:56:28.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:56:31.358 WARNING:teuthology.provision.fog.smithi064:[Errno None] Unable to connect to port 22 on 172.21.15.64
2024-03-13T22:56:37.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:56:40.414 WARNING:teuthology.provision.fog.smithi064:[Errno None] Unable to connect to port 22 on 172.21.15.64
2024-03-13T22:56:46.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:56:49.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:56:49.470 WARNING:teuthology.provision.fog.smithi064:[Errno None] Unable to connect to port 22 on 172.21.15.64
2024-03-13T22:56:52.446 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:56:53.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:56:55.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:56:56.514 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-03-13T22:56:58.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:56:58.527 WARNING:teuthology.provision.fog.smithi064:[Errno None] Unable to connect to port 22 on 172.21.15.64
2024-03-13T22:57:01.502 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:57:02.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:04.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:05.566 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-03-13T22:57:07.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:07.587 WARNING:teuthology.provision.fog.smithi064:[Errno None] Unable to connect to port 22 on 172.21.15.64
2024-03-13T22:57:10.558 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:57:11.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:13.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:14.622 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-03-13T22:57:16.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:16.670 WARNING:teuthology.provision.fog.smithi064:[Errno None] Unable to connect to port 22 on 172.21.15.64
2024-03-13T22:57:19.614 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:57:20.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:22.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:23.678 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-03-13T22:57:25.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:28.674 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:57:29.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:32.734 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-03-13T22:57:34.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:37.758 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:57:38.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:41.790 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-03-13T22:57:43.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:46.814 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:57:47.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:50.846 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-03-13T22:57:52.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:55.874 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:57:56.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:57:59.902 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-03-13T22:58:01.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:58:04.958 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-03-13T22:58:05.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:58:10.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:58:22.687 WARNING:teuthology.provision.fog.smithi064:timed out
2024-03-13T22:58:28.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:58:28.979 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-13T22:58:28.979 DEBUG:teuthology.orchestra.run.smithi064:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-13T22:58:29.307 INFO:teuthology.provision.fog.smithi064:Node is ready
2024-03-13T22:58:29.308 DEBUG:teuthology.orchestra.run.smithi064:> hostname
2024-03-13T22:58:29.324 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064.front.sepia.ceph.com
2024-03-13T22:58:29.324 DEBUG:teuthology.orchestra.run.smithi064:> grep smithi064.front.sepia.ceph.com /etc/hosts
2024-03-13T22:58:29.381 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64 smithi064.front.sepia.ceph.com smithi064
2024-03-13T22:58:29.381 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname smithi064
2024-03-13T22:58:29.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i -e 's/smithi064.front.sepia.ceph.com/smithi064/g' /etc/hosts
2024-03-13T22:58:29.528 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i -e 's/172.21.15.64/172.21.15.64/g' /etc/hosts
2024-03-13T22:58:29.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i -e 's/smithi064.front.sepia.ceph.com/smithi064/g' /etc/hostname
2024-03-13T22:58:29.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname smithi064
2024-03-13T22:58:29.743 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release
2024-03-13T22:58:29.798 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-13T22:58:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-13T22:58:29.800 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-13T22:58:29.800 INFO:teuthology.provision.fog.smithi064:Deploy complete!
2024-03-13T22:58:29.802 DEBUG:teuthology.parallel:result is None
2024-03-13T22:59:05.943 WARNING:teuthology.provision.fog.smithi163:timed out
2024-03-13T22:59:10.966 WARNING:teuthology.provision.fog.smithi023:timed out
2024-03-13T22:59:11.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:59:12.416 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-13T22:59:12.417 DEBUG:teuthology.orchestra.run.smithi163:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-13T22:59:12.754 INFO:teuthology.provision.fog.smithi163:Node is ready
2024-03-13T22:59:12.754 DEBUG:teuthology.orchestra.run.smithi163:> hostname
2024-03-13T22:59:12.770 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163.front.sepia.ceph.com
2024-03-13T22:59:12.770 DEBUG:teuthology.orchestra.run.smithi163:> grep smithi163.front.sepia.ceph.com /etc/hosts
2024-03-13T22:59:12.826 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163
2024-03-13T22:59:12.826 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname smithi163
2024-03-13T22:59:12.927 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163.front.sepia.ceph.com/smithi163/g' /etc/hosts
2024-03-13T22:59:12.956 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/172.21.15.163/172.21.15.163/g' /etc/hosts
2024-03-13T22:59:13.024 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163.front.sepia.ceph.com/smithi163/g' /etc/hostname
2024-03-13T22:59:13.092 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname smithi163
2024-03-13T22:59:13.159 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release
2024-03-13T22:59:13.213 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-13T22:59:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-13T22:59:13.215 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-13T22:59:13.215 INFO:teuthology.provision.fog.smithi163:Deploy complete!
2024-03-13T22:59:13.215 DEBUG:teuthology.parallel:result is None
2024-03-13T22:59:16.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:59:17.196 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-13T22:59:17.196 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-13T22:59:17.531 INFO:teuthology.provision.fog.smithi023:Node is ready
2024-03-13T22:59:17.532 DEBUG:teuthology.orchestra.run.smithi023:> hostname
2024-03-13T22:59:17.546 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com
2024-03-13T22:59:17.547 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts
2024-03-13T22:59:17.602 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023
2024-03-13T22:59:17.603 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2024-03-13T22:59:17.731 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts
2024-03-13T22:59:17.763 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts
2024-03-13T22:59:17.834 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname
2024-03-13T22:59:17.906 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2024-03-13T22:59:17.976 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2024-03-13T22:59:18.031 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream"
2024-03-13T22:59:18.031 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9"
2024-03-13T22:59:18.031 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos"
2024-03-13T22:59:18.031 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora"
2024-03-13T22:59:18.031 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-13T22:59:18.032 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-13T22:59:18.032 INFO:teuthology.provision.fog.smithi023:Deploy complete!
2024-03-13T22:59:18.033 DEBUG:teuthology.parallel:result is None
2024-03-13T22:59:18.270 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com
2024-03-13T22:59:18.271 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-13T22:59:18.295 INFO:teuthology.lock.ops:Checking smithi064.front.sepia.ceph.com
2024-03-13T22:59:18.295 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-13T22:59:18.311 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com
2024-03-13T22:59:18.311 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-13T22:59:18.329 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server
2024-03-13T22:59:18.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com'
2024-03-13T22:59:18.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:59:18.469 DEBUG:teuthology.orchestra.run.smithi023:> true
2024-03-13T22:59:18.536 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com'
2024-03-13T22:59:18.537 DEBUG:teuthology.orchestra.run.smithi023:> uname -m
2024-03-13T22:59:18.591 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64
2024-03-13T22:59:18.591 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2024-03-13T22:59:18.646 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream"
2024-03-13T22:59:18.646 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9"
2024-03-13T22:59:18.646 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos"
2024-03-13T22:59:18.646 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora"
2024-03-13T22:59:18.646 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9"
2024-03-13T22:59:18.646 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9"
2024-03-13T22:59:18.647 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-13T22:59:18.647 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31"
2024-03-13T22:59:18.647 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon"
2024-03-13T22:59:18.647 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-13T22:59:18.647 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/"
2024-03-13T22:59:18.647 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-13T22:59:18.647 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-13T22:59:18.647 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-13T22:59:18.648 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2024-03-13T22:59:18.678 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: set os type and version on server
2024-03-13T22:59:18.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com'
2024-03-13T22:59:18.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:59:18.781 DEBUG:teuthology.orchestra.run.smithi064:> true
2024-03-13T22:59:18.965 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com'
2024-03-13T22:59:18.965 DEBUG:teuthology.orchestra.run.smithi064:> uname -m
2024-03-13T22:59:19.019 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64
2024-03-13T22:59:19.020 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release
2024-03-13T22:59:19.075 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream"
2024-03-13T22:59:19.075 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9"
2024-03-13T22:59:19.075 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-13T22:59:19.076 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-13T22:59:19.077 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server
2024-03-13T22:59:19.095 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server
2024-03-13T22:59:19.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com'
2024-03-13T22:59:19.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-13T22:59:19.184 DEBUG:teuthology.orchestra.run.smithi163:> true
2024-03-13T22:59:19.259 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com'
2024-03-13T22:59:19.259 DEBUG:teuthology.orchestra.run.smithi163:> uname -m
2024-03-13T22:59:19.314 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64
2024-03-13T22:59:19.314 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-13T22:59:19.368 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/"
2024-03-13T22:59:19.369 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-13T22:59:19.369 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-13T22:59:19.369 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-13T22:59:19.369 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server
2024-03-13T22:59:19.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T22:59:19.484 INFO:teuthology.dispatcher.supervisor:Running job 7597996
2024-03-13T22:59:19.485 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-13_19:25:03-rados-wip-yuri6-testing-2024-03-12-0858-distro-default-smithi/7597996 --name yuriw-2024-03-13_19:25:03-rados-wip-yuri6-testing-2024-03-12-0858-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} -- /home/teuthworker/archive/yuriw-2024-03-13_19:25:03-rados-wip-yuri6-testing-2024-03-12-0858-distro-default-smithi/7597996/orig.config.yaml
2024-03-13T22:59:19.488 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-03-13_19:25:03-rados-wip-yuri6-testing-2024-03-12-0858-distro-default-smithi/7597996
2024-03-13T22:59:19.488 INFO:teuthology.dispatcher.supervisor:Job PID: 707523
2024-03-13T22:59:19.488 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-13T23:01:19.529 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T23:03:19.698 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T23:05:19.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T23:07:20.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T23:09:20.226 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T23:11:20.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T23:13:20.571 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T23:15:20.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-13T23:15:20.850 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1