2023-07-21T07:02:37.252 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T07:02:37.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:02:37.254 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:02:52.917 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f52ae15b8b0>, signals=[15]) 2023-07-21T07:02:52.919 INFO:teuthology.lock.ops:Start node 'smithi101.front.sepia.ceph.com' reimaging 2023-07-21T07:02:52.919 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T07:02:52.920 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-07-21T07:03:02.953 INFO:teuthology.lock.ops:Start node 'smithi132.front.sepia.ceph.com' reimaging 2023-07-21T07:03:02.953 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T07:03:02.953 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-07-21T07:03:02.955 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /host {"name": "smithi101"} 2023-07-21T07:03:12.826 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /host {"name": "smithi132"} 2023-07-21T07:03:18.012 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-21T07:03:23.008 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-21T07:03:23.055 DEBUG:teuthology.provision.fog.smithi132:Requesting image smithi_centos_9.stream (ID 10) 2023-07-21T07:03:23.055 DEBUG:teuthology.provision.fog.smithi132:FOG request: PUT /host/237 {"imageID": 10} 2023-07-21T07:03:23.064 DEBUG:teuthology.provision.fog.smithi101:Requesting image smithi_centos_9.stream (ID 10) 2023-07-21T07:03:23.064 DEBUG:teuthology.provision.fog.smithi101:FOG request: PUT /host/206 {"imageID": 10} 2023-07-21T07:03:23.112 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of centos 9.stream 2023-07-21T07:03:23.113 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:03:23.115 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of centos 9.stream 2023-07-21T07:03:23.115 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-21T07:03:23.175 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T07:03:23.177 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T07:03:23.211 DEBUG:teuthology.provision.fog.smithi132:FOG request: POST /host/237/task {"taskTypeID": 1} 2023-07-21T07:03:23.221 DEBUG:teuthology.provision.fog.smithi101:FOG request: POST /host/206/task {"taskTypeID": 1} 2023-07-21T07:03:23.268 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:03:23.277 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-21T07:03:33.063 INFO:teuthology.orchestra.console:Power off smithi132 2023-07-21T07:03:33.064 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T07:03:33.081 INFO:teuthology.orchestra.console:Power off smithi101 2023-07-21T07:03:33.081 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T07:03:33.092 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:33.105 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:33.227 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2023-07-21T07:03:33.328 INFO:teuthology.orchestra.console:Power on smithi101 2023-07-21T07:03:33.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T07:03:33.349 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:37.118 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:37.372 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:41.228 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:41.438 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2023-07-21T07:03:41.482 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:41.540 INFO:teuthology.orchestra.console:Power on smithi132 2023-07-21T07:03:41.540 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T07:03:46.563 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:46.593 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:51.578 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:51.703 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:51.788 INFO:teuthology.orchestra.console:Power on for smithi132 completed 2023-07-21T07:03:51.889 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2023-07-21T07:03:51.890 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:03:57.131 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:03:57.340 INFO:teuthology.orchestra.console:Power on for smithi101 completed 2023-07-21T07:03:57.441 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish 2023-07-21T07:03:57.442 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-21T07:04:02.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:04:08.218 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:04:13.158 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-21T07:04:19.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:04:27.216 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-21T07:04:28.246 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:04:33.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:04:36.304 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-21T07:04:42.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:04:48.319 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:05:05.648 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-21T07:05:08.394 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:05:13.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:05:14.694 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-21T07:05:20.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:05:25.779 ERROR:teuthology.orchestra.connection:Error authenticating with smithi101.front.sepia.ceph.com: Authentication failed. 2023-07-21T07:05:25.781 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T07:05:25.782 DEBUG:teuthology.orchestra.run.smithi101:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T07:05:25.783 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:26.787 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:27.791 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:28.315 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:05:33.321 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:34.326 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:35.331 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:36.336 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:37.342 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:38.347 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:39.352 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:40.357 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:41.362 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:42.368 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:43.374 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:44.379 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:45.385 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:46.390 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:47.395 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:48.390 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:05:53.398 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:54.403 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:55.408 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:56.414 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:57.419 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:58.424 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:05:59.430 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:06:00.436 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:06:01.441 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:06:02.446 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:06:03.451 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:06:04.456 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:06:05.462 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:06:06.467 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T07:06:08.470 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:06:23.903 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-21T07:06:28.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:06:32.048 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-21T07:06:38.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:06:41.104 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-21T07:06:47.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:06:50.160 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-21T07:06:56.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:06:59.216 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-21T07:07:05.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:07:13.296 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-21T07:07:19.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:07:54.864 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-21T07:07:54.866 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi101 2023-07-21T07:08:03.875 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T07:08:03.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:08:04.143 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-07-21T07:08:04.470 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T07:08:04.470 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T07:08:04.471 DEBUG:teuthology.orchestra.run.smithi101:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T07:08:09.486 INFO:teuthology.provision.fog.smithi101:Node is ready 2023-07-21T07:08:09.487 DEBUG:teuthology.orchestra.run.smithi101:> hostname 2023-07-21T07:08:09.504 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com 2023-07-21T07:08:09.505 DEBUG:teuthology.orchestra.run.smithi101:> grep smithi101.front.sepia.ceph.com /etc/hosts 2023-07-21T07:08:09.562 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101 2023-07-21T07:08:09.562 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname smithi101 2023-07-21T07:08:09.689 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/smithi101.front.sepia.ceph.com/smithi101/g' /etc/hosts 2023-07-21T07:08:09.720 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/172.21.15.101/172.21.15.101/g' /etc/hosts 2023-07-21T07:08:09.789 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/smithi101.front.sepia.ceph.com/smithi101/g' /etc/hostname 2023-07-21T07:08:09.858 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname smithi101 2023-07-21T07:08:09.926 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-07-21T07:08:09.980 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="CentOS Stream" 2023-07-21T07:08:09.981 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="9" 2023-07-21T07:08:09.981 INFO:teuthology.orchestra.run.smithi101.stdout:ID="centos" 2023-07-21T07:08:09.981 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="rhel fedora" 2023-07-21T07:08:09.981 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="9" 2023-07-21T07:08:09.981 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el9" 2023-07-21T07:08:09.981 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T07:08:09.982 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-07-21T07:08:09.982 INFO:teuthology.orchestra.run.smithi101.stdout:LOGO="fedora-logo-icon" 2023-07-21T07:08:09.982 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T07:08:09.982 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://centos.org/" 2023-07-21T07:08:09.982 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T07:08:09.982 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T07:08:09.982 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T07:08:09.983 INFO:teuthology.provision.fog.smithi101:Deploy complete! 2023-07-21T07:08:09.984 DEBUG:teuthology.parallel:result is None 2023-07-21T07:08:19.300 WARNING:teuthology.provision.fog.smithi132:timed out 2023-07-21T07:08:25.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:08:30.571 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T07:08:30.572 DEBUG:teuthology.orchestra.run.smithi132:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T07:08:30.873 INFO:teuthology.provision.fog.smithi132:Node is ready 2023-07-21T07:08:30.874 DEBUG:teuthology.orchestra.run.smithi132:> hostname 2023-07-21T07:08:30.888 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com 2023-07-21T07:08:30.888 DEBUG:teuthology.orchestra.run.smithi132:> grep smithi132.front.sepia.ceph.com /etc/hosts 2023-07-21T07:08:30.942 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132 2023-07-21T07:08:30.943 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-07-21T07:08:31.056 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132.front.sepia.ceph.com/smithi132/g' /etc/hosts 2023-07-21T07:08:31.084 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/172.21.15.132/172.21.15.132/g' /etc/hosts 2023-07-21T07:08:31.150 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132.front.sepia.ceph.com/smithi132/g' /etc/hostname 2023-07-21T07:08:31.218 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-07-21T07:08:31.284 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-07-21T07:08:31.338 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2023-07-21T07:08:31.338 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2023-07-21T07:08:31.338 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2023-07-21T07:08:31.338 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2023-07-21T07:08:31.338 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2023-07-21T07:08:31.339 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2023-07-21T07:08:31.339 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T07:08:31.339 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-07-21T07:08:31.339 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2023-07-21T07:08:31.339 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T07:08:31.339 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2023-07-21T07:08:31.339 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T07:08:31.339 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T07:08:31.340 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T07:08:31.340 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2023-07-21T07:08:31.340 DEBUG:teuthology.parallel:result is None 2023-07-21T07:08:51.554 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com 2023-07-21T07:08:51.555 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T07:08:56.586 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com 2023-07-21T07:08:56.587 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T07:09:01.629 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server 2023-07-21T07:09:01.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T07:09:01.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:09:06.764 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-07-21T07:09:06.934 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T07:09:06.935 DEBUG:teuthology.orchestra.run.smithi101:> uname -m 2023-07-21T07:09:06.989 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2023-07-21T07:09:06.990 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-07-21T07:09:07.043 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="CentOS Stream" 2023-07-21T07:09:07.044 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="9" 2023-07-21T07:09:07.044 INFO:teuthology.orchestra.run.smithi101.stdout:ID="centos" 2023-07-21T07:09:07.044 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="rhel fedora" 2023-07-21T07:09:07.044 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="9" 2023-07-21T07:09:07.044 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el9" 2023-07-21T07:09:07.044 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T07:09:07.045 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-07-21T07:09:07.045 INFO:teuthology.orchestra.run.smithi101.stdout:LOGO="fedora-logo-icon" 2023-07-21T07:09:07.045 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T07:09:07.045 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://centos.org/" 2023-07-21T07:09:07.045 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T07:09:07.045 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T07:09:07.045 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T07:09:07.046 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-07-21T07:09:12.076 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server 2023-07-21T07:09:12.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-07-21T07:09:12.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T07:09:17.164 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-07-21T07:09:17.293 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-07-21T07:09:17.294 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-07-21T07:09:17.349 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-07-21T07:09:17.350 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-07-21T07:09:17.409 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2023-07-21T07:09:17.411 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2023-07-21T07:09:17.411 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2023-07-21T07:09:17.412 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2023-07-21T07:09:17.413 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2023-07-21T07:09:17.413 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2023-07-21T07:09:17.413 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T07:09:17.414 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-07-21T07:09:17.414 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2023-07-21T07:09:17.415 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T07:09:17.415 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2023-07-21T07:09:17.416 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T07:09:17.416 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T07:09:17.416 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T07:09:17.418 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-07-21T07:09:22.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:09:22.449 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:09:27.187 INFO:teuthology.dispatcher.supervisor:Running job 7346407 2023-07-21T07:09:27.187 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346407 --name vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi --description fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/iozone}}} -- /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346407/orig.config.yaml 2023-07-21T07:09:27.192 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346407 2023-07-21T07:09:27.193 INFO:teuthology.dispatcher.supervisor:Job PID: 19265 2023-07-21T07:09:27.193 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-21T07:11:27.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:11:27.253 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:13:32.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:13:32.441 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:13:37.523 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-21T07:13:44.564 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-21T07:13:44.564 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T07:13:54.630 DEBUG:teuthology.nuke:shortname: smithi101 2023-07-21T07:13:54.631 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T07:13:54.660 DEBUG:teuthology.nuke:shortname: smithi132 2023-07-21T07:13:54.660 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T07:13:54.663 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 07:02:36.508791', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPVBonqacJFNGjM70i4ruXlzx0OmedylgTCh3JR2OsnfOYyw0erZdSgMNra8DjaBuza93BqOncHorMeuRLhBvQY='} 2023-07-21T07:13:54.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 07:02:36.510258', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGX/JQ4Ta1jYd6FQc4GscCavjf8r3zYaXTepYh64rh7ZyVVq2BqA5DUu6ZeK1S1RKaI77hD4rczLU9Gi7NYOo10='} 2023-07-21T07:13:54.714 INFO:teuthology.orchestra.console:Power off smithi101 2023-07-21T07:13:54.715 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T07:13:54.737 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:13:54.752 INFO:teuthology.orchestra.console:Power off smithi132 2023-07-21T07:13:54.753 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T07:13:54.773 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:13:54.862 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2023-07-21T07:13:54.894 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2023-07-21T07:14:08.277 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com 2023-07-21T07:14:08.287 INFO:teuthology.lock.ops:unlocked: smithi101.front.sepia.ceph.com