2023-07-14T21:37:59.376 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T21:37:59.377 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:37:59.377 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:37:59.558 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2d03ba88b0>, signals=[15]) 2023-07-14T21:37:59.559 INFO:teuthology.lock.ops:Start node 'smithi099.front.sepia.ceph.com' reimaging 2023-07-14T21:37:59.560 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T21:37:59.560 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-07-14T21:37:59.582 INFO:teuthology.lock.ops:Start node 'smithi101.front.sepia.ceph.com' reimaging 2023-07-14T21:37:59.583 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T21:37:59.583 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-07-14T21:37:59.584 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /host {"name": "smithi099"} 2023-07-14T21:37:59.607 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging 2023-07-14T21:37:59.607 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T21:37:59.608 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-07-14T21:37:59.609 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /host {"name": "smithi101"} 2023-07-14T21:37:59.633 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /host {"name": "smithi181"} 2023-07-14T21:37:59.659 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T21:37:59.679 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T21:37:59.693 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T21:37:59.698 DEBUG:teuthology.provision.fog.smithi099:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T21:37:59.698 DEBUG:teuthology.provision.fog.smithi099:FOG request: PUT /host/204 {"imageID": 10} 2023-07-14T21:37:59.717 DEBUG:teuthology.provision.fog.smithi101:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T21:37:59.718 DEBUG:teuthology.provision.fog.smithi101:FOG request: PUT /host/206 {"imageID": 10} 2023-07-14T21:37:59.732 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T21:37:59.732 DEBUG:teuthology.provision.fog.smithi181:FOG request: PUT /host/286 {"imageID": 10} 2023-07-14T21:37:59.741 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of centos 9.stream 2023-07-14T21:37:59.741 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-07-14T21:37:59.755 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of centos 9.stream 2023-07-14T21:37:59.755 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:37:59.775 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 9.stream 2023-07-14T21:37:59.775 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:37:59.813 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T21:37:59.817 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T21:37:59.854 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T21:37:59.857 DEBUG:teuthology.provision.fog.smithi099:FOG request: POST /host/204/task {"taskTypeID": 1} 2023-07-14T21:37:59.865 DEBUG:teuthology.provision.fog.smithi101:FOG request: POST /host/206/task {"taskTypeID": 1} 2023-07-14T21:37:59.890 DEBUG:teuthology.provision.fog.smithi181:FOG request: POST /host/286/task {"taskTypeID": 1} 2023-07-14T21:37:59.905 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-07-14T21:37:59.911 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:37:59.947 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:38:00.007 INFO:teuthology.orchestra.console:Power off smithi099 2023-07-14T21:38:00.009 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T21:38:00.035 INFO:teuthology.orchestra.console:Power off smithi101 2023-07-14T21:38:00.036 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T21:38:00.048 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:00.065 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:00.083 INFO:teuthology.orchestra.console:Power off smithi181 2023-07-14T21:38:00.084 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T21:38:00.105 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:04.096 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:04.106 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:04.129 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:09.209 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:09.217 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:09.239 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:09.420 INFO:teuthology.orchestra.console:Power off for smithi099 completed 2023-07-14T21:38:09.521 INFO:teuthology.orchestra.console:Power on smithi099 2023-07-14T21:38:09.521 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T21:38:09.543 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:13.569 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:14.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:14.350 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:14.539 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2023-07-14T21:38:14.640 INFO:teuthology.orchestra.console:Power on smithi101 2023-07-14T21:38:14.640 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T21:38:14.661 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:18.681 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:18.690 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:19.461 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:19.672 INFO:teuthology.orchestra.console:Power off for smithi181 completed 2023-07-14T21:38:19.773 INFO:teuthology.orchestra.console:Power on smithi181 2023-07-14T21:38:19.774 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T21:38:19.794 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:23.791 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:23.803 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:23.815 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:28.903 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:28.913 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:28.926 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:34.014 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:34.024 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:34.034 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:34.236 INFO:teuthology.orchestra.console:Power on for smithi101 completed 2023-07-14T21:38:34.246 INFO:teuthology.orchestra.console:Power on for smithi181 completed 2023-07-14T21:38:34.338 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish 2023-07-14T21:38:34.338 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:38:34.347 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish 2023-07-14T21:38:34.348 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:38:39.128 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:44.243 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:49.355 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T21:38:49.422 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:38:54.429 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:39:09.541 INFO:teuthology.orchestra.console:Power on for smithi099 completed 2023-07-14T21:39:09.642 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish 2023-07-14T21:39:09.643 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-07-14T21:39:24.505 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:39:24.506 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:39:29.720 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-07-14T21:39:39.900 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:39:39.902 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:39:45.016 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-07-14T21:39:54.984 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:39:55.088 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:40:05.092 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-07-14T21:40:10.152 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:40:15.154 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:40:20.185 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-07-14T21:40:40.208 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:40:40.210 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T21:40:50.216 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-07-14T21:40:55.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:00.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:05.308 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-14T21:41:08.372 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-07-14T21:41:10.224 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-14T21:41:15.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:20.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:25.241 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-14T21:41:25.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:30.309 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-07-14T21:41:31.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:36.247 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-14T21:41:36.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:39.312 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-14T21:41:39.376 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-07-14T21:41:42.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:47.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:52.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:41:57.268 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-14T21:42:03.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:42:21.808 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-14T21:42:27.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:42:29.048 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T21:42:29.050 DEBUG:teuthology.orchestra.run.smithi099:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T21:42:30.864 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-14T21:42:36.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:42:42.069 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T21:42:42.070 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T21:42:52.263 WARNING:teuthology.provision.fog.smithi101:timed out 2023-07-14T21:42:54.402 INFO:teuthology.provision.fog.smithi099:Node is ready 2023-07-14T21:42:54.403 DEBUG:teuthology.orchestra.run.smithi099:> hostname 2023-07-14T21:42:54.421 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com 2023-07-14T21:42:54.422 DEBUG:teuthology.orchestra.run.smithi099:> grep smithi099.front.sepia.ceph.com /etc/hosts 2023-07-14T21:42:54.479 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099 2023-07-14T21:42:54.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099 2023-07-14T21:42:54.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hosts 2023-07-14T21:42:54.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/172.21.15.99/172.21.15.99/g' /etc/hosts 2023-07-14T21:42:54.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hostname 2023-07-14T21:42:54.795 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099 2023-07-14T21:42:54.863 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-07-14T21:42:54.917 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2023-07-14T21:42:54.918 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2023-07-14T21:42:54.918 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2023-07-14T21:42:54.918 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2023-07-14T21:42:54.918 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2023-07-14T21:42:54.918 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2023-07-14T21:42:54.918 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T21:42:54.919 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-07-14T21:42:54.919 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2023-07-14T21:42:54.919 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T21:42:54.919 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2023-07-14T21:42:54.919 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T21:42:54.919 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T21:42:54.919 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T21:42:54.920 INFO:teuthology.provision.fog.smithi099:Deploy complete! 2023-07-14T21:42:54.921 DEBUG:teuthology.parallel:result is None 2023-07-14T21:42:58.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:42:58.570 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T21:42:58.571 DEBUG:teuthology.orchestra.run.smithi101:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T21:42:58.897 INFO:teuthology.provision.fog.smithi101:Node is ready 2023-07-14T21:42:58.897 DEBUG:teuthology.orchestra.run.smithi101:> hostname 2023-07-14T21:42:58.917 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com 2023-07-14T21:42:58.918 DEBUG:teuthology.orchestra.run.smithi101:> grep smithi101.front.sepia.ceph.com /etc/hosts 2023-07-14T21:42:58.977 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101 2023-07-14T21:42:58.977 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname smithi101 2023-07-14T21:42:59.100 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/smithi101.front.sepia.ceph.com/smithi101/g' /etc/hosts 2023-07-14T21:42:59.127 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/172.21.15.101/172.21.15.101/g' /etc/hosts 2023-07-14T21:42:59.198 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/smithi101.front.sepia.ceph.com/smithi101/g' /etc/hostname 2023-07-14T21:42:59.269 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname smithi101 2023-07-14T21:42:59.339 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-07-14T21:42:59.398 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="CentOS Stream" 2023-07-14T21:42:59.399 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="9" 2023-07-14T21:42:59.399 INFO:teuthology.orchestra.run.smithi101.stdout:ID="centos" 2023-07-14T21:42:59.399 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="rhel fedora" 2023-07-14T21:42:59.400 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="9" 2023-07-14T21:42:59.400 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el9" 2023-07-14T21:42:59.400 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T21:42:59.401 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-07-14T21:42:59.401 INFO:teuthology.orchestra.run.smithi101.stdout:LOGO="fedora-logo-icon" 2023-07-14T21:42:59.401 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T21:42:59.402 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://centos.org/" 2023-07-14T21:42:59.402 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T21:42:59.402 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T21:42:59.403 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T21:42:59.403 INFO:teuthology.provision.fog.smithi101:Deploy complete! 2023-07-14T21:42:59.404 DEBUG:teuthology.parallel:result is None 2023-07-14T21:43:02.403 INFO:teuthology.provision.fog.smithi181:Node is ready 2023-07-14T21:43:02.404 DEBUG:teuthology.orchestra.run.smithi181:> hostname 2023-07-14T21:43:02.423 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com 2023-07-14T21:43:02.424 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts 2023-07-14T21:43:02.481 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181 2023-07-14T21:43:02.482 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-07-14T21:43:02.660 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts 2023-07-14T21:43:02.692 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts 2023-07-14T21:43:02.763 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname 2023-07-14T21:43:02.834 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-07-14T21:43:02.902 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-07-14T21:43:02.957 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-07-14T21:43:02.957 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2023-07-14T21:43:02.957 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-07-14T21:43:02.957 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-07-14T21:43:02.958 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2023-07-14T21:43:02.958 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2023-07-14T21:43:02.958 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T21:43:02.958 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-07-14T21:43:02.958 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2023-07-14T21:43:02.958 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T21:43:02.958 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-07-14T21:43:02.959 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T21:43:02.959 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T21:43:02.959 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T21:43:02.959 INFO:teuthology.provision.fog.smithi181:Deploy complete! 2023-07-14T21:43:02.960 DEBUG:teuthology.parallel:result is None 2023-07-14T21:43:05.230 INFO:teuthology.lock.ops:Checking smithi099.front.sepia.ceph.com 2023-07-14T21:43:05.231 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-14T21:43:05.256 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com 2023-07-14T21:43:05.257 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-14T21:43:05.281 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com 2023-07-14T21:43:05.281 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-14T21:43:05.308 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: set os type and version on server 2023-07-14T21:43:05.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-07-14T21:43:05.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:43:05.391 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-07-14T21:43:05.559 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-07-14T21:43:05.559 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2023-07-14T21:43:05.614 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2023-07-14T21:43:05.615 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-07-14T21:43:05.670 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2023-07-14T21:43:05.670 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2023-07-14T21:43:05.671 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2023-07-14T21:43:05.671 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2023-07-14T21:43:05.671 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2023-07-14T21:43:05.672 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2023-07-14T21:43:05.672 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T21:43:05.672 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-07-14T21:43:05.672 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2023-07-14T21:43:05.673 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T21:43:05.673 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2023-07-14T21:43:05.673 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T21:43:05.674 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T21:43:05.674 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T21:43:05.675 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-07-14T21:43:05.698 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server 2023-07-14T21:43:05.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-14T21:43:05.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:43:05.780 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-07-14T21:43:05.846 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-14T21:43:05.847 DEBUG:teuthology.orchestra.run.smithi101:> uname -m 2023-07-14T21:43:05.900 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2023-07-14T21:43:05.901 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-07-14T21:43:05.954 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="CentOS Stream" 2023-07-14T21:43:05.954 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="9" 2023-07-14T21:43:05.954 INFO:teuthology.orchestra.run.smithi101.stdout:ID="centos" 2023-07-14T21:43:05.954 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="rhel fedora" 2023-07-14T21:43:05.955 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="9" 2023-07-14T21:43:05.955 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el9" 2023-07-14T21:43:05.955 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T21:43:05.955 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-07-14T21:43:05.955 INFO:teuthology.orchestra.run.smithi101.stdout:LOGO="fedora-logo-icon" 2023-07-14T21:43:05.955 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T21:43:05.955 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://centos.org/" 2023-07-14T21:43:05.955 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T21:43:05.956 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T21:43:05.956 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T21:43:05.956 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-07-14T21:43:05.981 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server 2023-07-14T21:43:05.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-14T21:43:05.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T21:43:06.064 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-07-14T21:43:06.136 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-14T21:43:06.136 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-07-14T21:43:06.191 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-07-14T21:43:06.192 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-07-14T21:43:06.248 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-07-14T21:43:06.248 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2023-07-14T21:43:06.248 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-07-14T21:43:06.248 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-07-14T21:43:06.249 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2023-07-14T21:43:06.249 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2023-07-14T21:43:06.249 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T21:43:06.249 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-07-14T21:43:06.249 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2023-07-14T21:43:06.249 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T21:43:06.249 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-07-14T21:43:06.249 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T21:43:06.250 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T21:43:06.250 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T21:43:06.250 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-07-14T21:43:06.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:43:06.274 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:43:06.366 INFO:teuthology.dispatcher.supervisor:Running job 7337843 2023-07-14T21:43:06.366 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7337843 --name yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi --description fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} -- /home/teuthworker/archive/yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7337843/orig.config.yaml 2023-07-14T21:43:06.369 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7337843 2023-07-14T21:43:06.370 INFO:teuthology.dispatcher.supervisor:Job PID: 27222 2023-07-14T21:43:06.371 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-14T21:45:06.463 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:45:06.465 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:47:07.689 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:47:07.690 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:49:07.853 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:49:07.854 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:51:13.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:51:13.149 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:53:18.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:53:18.322 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:55:23.496 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:55:23.497 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:57:27.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T21:57:27.205 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T21:57:32.287 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120