2023-07-27T13:30:45.420 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-27T13:30:45.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:30:45.422 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:31:09.799 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f08bd80e430>, signals=[15]) 2023-07-27T13:31:09.803 INFO:teuthology.lock.ops:Start node 'smithi142.front.sepia.ceph.com' reimaging 2023-07-27T13:31:09.804 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: reset os type and version on server 2023-07-27T13:31:09.804 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2023-07-27T13:31:14.838 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging 2023-07-27T13:31:14.838 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server 2023-07-27T13:31:14.839 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-07-27T13:31:14.840 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /host {"name": "smithi142"} 2023-07-27T13:31:19.879 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /host {"name": "smithi181"} 2023-07-27T13:31:29.925 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-27T13:31:30.035 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-27T13:31:30.087 DEBUG:teuthology.provision.fog.smithi142:Requesting image smithi_centos_8.stream (ID 3) 2023-07-27T13:31:30.087 DEBUG:teuthology.provision.fog.smithi142:FOG request: PUT /host/247 {"imageID": 3} 2023-07-27T13:31:30.090 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_centos_8.stream (ID 3) 2023-07-27T13:31:30.090 DEBUG:teuthology.provision.fog.smithi181:FOG request: PUT /host/286 {"imageID": 3} 2023-07-27T13:31:30.130 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 8.stream 2023-07-27T13:31:30.130 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-27T13:31:30.133 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of centos 8.stream 2023-07-27T13:31:30.133 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:31:30.175 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /tasktype {"name": "deploy"} 2023-07-27T13:31:35.182 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /tasktype {"name": "deploy"} 2023-07-27T13:31:35.689 DEBUG:teuthology.provision.fog.smithi142:FOG request: POST /host/247/task {"taskTypeID": 1} 2023-07-27T13:31:35.697 DEBUG:teuthology.provision.fog.smithi181:FOG request: POST /host/286/task {"taskTypeID": 1} 2023-07-27T13:31:35.750 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:31:35.753 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-27T13:31:45.844 INFO:teuthology.orchestra.console:Power off smithi181 2023-07-27T13:31:45.845 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T13:31:45.864 INFO:teuthology.orchestra.console:Power off smithi142 2023-07-27T13:31:45.865 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T13:31:45.874 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T13:31:45.875 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T13:31:45.886 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T13:31:45.887 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T13:31:45.896 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T13:31:45.907 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T13:31:45.998 INFO:teuthology.orchestra.console:Power off for smithi181 completed 2023-07-27T13:31:46.008 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2023-07-27T13:31:46.099 INFO:teuthology.orchestra.console:Power on smithi181 2023-07-27T13:31:46.100 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-27T13:31:46.110 INFO:teuthology.orchestra.console:Power on smithi142 2023-07-27T13:31:46.110 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-27T13:31:46.120 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-27T13:31:46.121 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T13:31:46.131 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-27T13:31:46.132 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T13:31:46.143 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T13:31:46.155 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T13:31:50.149 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T13:31:50.158 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T13:31:50.269 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T13:31:50.370 INFO:teuthology.orchestra.console:Power on for smithi142 completed 2023-07-27T13:31:50.471 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish 2023-07-27T13:31:50.471 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:31:51.261 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T13:31:55.262 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T13:31:55.372 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T13:31:55.473 INFO:teuthology.orchestra.console:Power on for smithi181 completed 2023-07-27T13:31:55.574 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish 2023-07-27T13:31:55.574 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-27T13:31:55.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:32:05.540 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:32:14.064 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-27T13:32:20.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:32:23.120 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-27T13:32:25.601 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:32:29.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:32:37.200 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-27T13:32:40.660 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:32:43.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:32:47.824 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-27T13:32:53.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:32:58.833 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:33:17.296 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-27T13:33:18.906 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:33:23.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:33:25.791 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-27T13:33:31.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:33:31.911 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: Authentication failed. 2023-07-27T13:33:31.913 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T13:33:31.913 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T13:33:31.914 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:32.920 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:33.924 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:33.967 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:33:37.621 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:38.627 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:39.631 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:40.636 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:41.642 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:42.647 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:43.652 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:44.656 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:45.661 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:46.666 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:47.670 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:48.675 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:49.680 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:50.685 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:51.690 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:52.695 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:52.700 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:33:57.706 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:58.711 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:33:59.715 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:00.721 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:01.726 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:02.731 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:03.736 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:04.741 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:05.746 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:06.752 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:07.757 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:08.762 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:09.767 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:10.772 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:11.778 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:12.783 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:12.805 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:34:17.814 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:18.819 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:19.824 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:20.829 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:21.834 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:22.839 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:23.845 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:24.851 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:25.856 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T13:34:32.926 DEBUG:teuthology.provision.fog.smithi142:FOG request: GET /task/active None 2023-07-27T13:34:38.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:34:41.104 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-07-27T13:34:47.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:34:50.160 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-07-27T13:34:56.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:35:04.240 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-07-27T13:35:10.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:35:13.296 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-07-27T13:35:19.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:35:27.376 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-07-27T13:35:33.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:36:33.379 WARNING:teuthology.provision.fog.smithi142:timed out 2023-07-27T13:36:39.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:36:44.387 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-27T13:36:44.388 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi181 2023-07-27T13:36:44.660 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T13:36:44.660 DEBUG:teuthology.orchestra.run.smithi142:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T13:36:44.919 INFO:teuthology.provision.fog.smithi142:Node is ready 2023-07-27T13:36:44.920 DEBUG:teuthology.orchestra.run.smithi142:> hostname 2023-07-27T13:36:44.940 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142.front.sepia.ceph.com 2023-07-27T13:36:44.941 DEBUG:teuthology.orchestra.run.smithi142:> grep smithi142.front.sepia.ceph.com /etc/hosts 2023-07-27T13:36:45.000 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142 smithi142.front.sepia.ceph.com smithi142 2023-07-27T13:36:45.001 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname smithi142 2023-07-27T13:36:45.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i -e 's/smithi142.front.sepia.ceph.com/smithi142/g' /etc/hosts 2023-07-27T13:36:45.123 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i -e 's/172.21.15.142/172.21.15.142/g' /etc/hosts 2023-07-27T13:36:45.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i -e 's/smithi142.front.sepia.ceph.com/smithi142/g' /etc/hostname 2023-07-27T13:36:45.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname smithi142 2023-07-27T13:36:45.326 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2023-07-27T13:36:45.384 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2023-07-27T13:36:45.384 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8" 2023-07-27T13:36:45.384 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2023-07-27T13:36:45.384 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2023-07-27T13:36:45.385 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8" 2023-07-27T13:36:45.385 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2023-07-27T13:36:45.385 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-27T13:36:45.385 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2023-07-27T13:36:45.385 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-27T13:36:45.385 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2023-07-27T13:36:45.385 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T13:36:45.386 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T13:36:45.386 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-27T13:36:45.386 INFO:teuthology.provision.fog.smithi142:Deploy complete! 2023-07-27T13:36:45.388 DEBUG:teuthology.parallel:result is None 2023-07-27T13:36:53.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-27T13:36:53.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:36:58.623 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-07-27T13:36:58.888 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-27T13:36:58.889 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T13:36:58.889 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T13:36:58.910 INFO:teuthology.provision.fog.smithi181:Node is ready 2023-07-27T13:36:58.910 DEBUG:teuthology.orchestra.run.smithi181:> hostname 2023-07-27T13:36:58.969 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com 2023-07-27T13:36:58.970 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts 2023-07-27T13:36:59.031 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181 2023-07-27T13:36:59.032 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-07-27T13:36:59.152 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts 2023-07-27T13:36:59.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts 2023-07-27T13:36:59.255 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname 2023-07-27T13:36:59.326 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-07-27T13:36:59.393 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-07-27T13:36:59.451 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-07-27T13:36:59.452 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2023-07-27T13:36:59.452 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-07-27T13:36:59.452 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-07-27T13:36:59.452 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2023-07-27T13:36:59.452 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-07-27T13:36:59.453 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-27T13:36:59.453 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-07-27T13:36:59.453 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-27T13:36:59.453 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-07-27T13:36:59.453 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T13:36:59.454 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T13:36:59.454 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-27T13:36:59.454 INFO:teuthology.provision.fog.smithi181:Deploy complete! 2023-07-27T13:36:59.455 DEBUG:teuthology.parallel:result is None 2023-07-27T13:37:14.627 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com 2023-07-27T13:37:14.628 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-27T13:37:19.657 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com 2023-07-27T13:37:19.657 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-27T13:37:24.690 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server 2023-07-27T13:37:24.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-07-27T13:37:24.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:37:24.763 DEBUG:teuthology.orchestra.run.smithi142:> true 2023-07-27T13:37:24.898 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-07-27T13:37:24.899 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2023-07-27T13:37:24.958 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2023-07-27T13:37:24.958 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2023-07-27T13:37:25.017 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2023-07-27T13:37:25.017 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8" 2023-07-27T13:37:25.017 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2023-07-27T13:37:25.017 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2023-07-27T13:37:25.017 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8" 2023-07-27T13:37:25.018 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2023-07-27T13:37:25.018 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-27T13:37:25.018 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2023-07-27T13:37:25.018 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-27T13:37:25.018 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2023-07-27T13:37:25.018 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T13:37:25.018 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T13:37:25.019 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-27T13:37:25.019 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2023-07-27T13:37:25.039 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server 2023-07-27T13:37:25.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-27T13:37:25.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T13:37:30.110 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-07-27T13:37:30.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-27T13:37:30.247 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-07-27T13:37:30.304 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-07-27T13:37:30.305 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-07-27T13:37:30.363 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-07-27T13:37:30.363 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2023-07-27T13:37:30.363 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-07-27T13:37:30.364 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-07-27T13:37:30.364 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2023-07-27T13:37:30.364 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-07-27T13:37:30.364 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-27T13:37:30.365 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-07-27T13:37:30.365 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-27T13:37:30.365 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-07-27T13:37:30.365 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T13:37:30.365 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T13:37:30.365 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-27T13:37:30.366 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-07-27T13:37:35.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:37:35.196 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:37:35.287 INFO:teuthology.dispatcher.supervisor:Running job 7354373 2023-07-27T13:37:35.287 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354373 --name yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} -- /home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354373/orig.config.yaml 2023-07-27T13:37:35.290 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354373 2023-07-27T13:37:35.291 INFO:teuthology.dispatcher.supervisor:Job PID: 21187 2023-07-27T13:37:35.291 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-27T13:39:35.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:39:35.393 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:41:40.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:41:40.558 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:43:43.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:43:43.705 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:45:48.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:45:48.889 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:47:54.072 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:47:54.073 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:49:59.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:49:59.261 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:52:04.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:52:04.450 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:54:09.633 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:54:09.634 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:56:14.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:56:14.817 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T13:58:16.657 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T13:58:16.658 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:00:21.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:00:21.836 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:02:27.005 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:02:27.006 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:04:32.177 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:04:32.178 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:04:37.252 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-27T14:04:40.865 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-27T14:04:40.865 INFO:teuthology.nuke:Checking targets against current locks 2023-07-27T14:04:40.948 DEBUG:teuthology.nuke:shortname: smithi142 2023-07-27T14:04:40.949 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-27T14:04:40.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354373', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-27 13:30:29.593594', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1Y9Rk433+th5+WQ9FRJsrsxMBWoMl0T7ACFJgLZlM+Zwsu3xgEhmLbty10upgRjrgT8geeX8WkZKITHwAhb5Kh1cdPO8A7InmhVDEfobn4LAgLpSRu+8DImJ5uB0vdzBoSMIsX9X+OfcL63eXJ2Gp/7XKZijEuSiEZwh+BA+1nV5ZGsBc0M0S+mlKSDbAUnDMmIoqY98l5KPaZMAvXDQ1UWx/GPZpseuEwzufKn8V9a+o8WQcKMey/wd7jxEJhY0n/ZPiS31+bvj226C4fpQAdTt2bb7+e2caSr36lszfF6WEVNWMLIrbAeQvsFh4JFs+VZ/8nwE5gMrBhNSShxy3jGeVb2x+cwsrWSmtnC+77nUssNgDM+cJCX/9u+rq3tyJWhQSPdynXnMUGHLxVXoV4o1B9f/ZKDhEP9FdS96B3jLPYv0Vy0H6Am5p+BNyGv3DGki+Imtwdv512CT3H2JDSWnv67zT1dnU2Dlrb+VAFKmxCJiyOfgtblmvrP0zBHU='} 2023-07-27T14:04:40.976 DEBUG:teuthology.nuke:shortname: smithi181 2023-07-27T14:04:40.977 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-27T14:04:41.003 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354373', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-27 13:30:29.594867', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzOl+DycpkDya8E2uhL6mjZ7d+Ws45gE2omhbgp9ZiivVo2XGOFBQ2rfE9KYQabJN1z3OZWWdSJyvM2A65GQQY='} 2023-07-27T14:04:41.028 INFO:teuthology.orchestra.console:Power off smithi142 2023-07-27T14:04:41.030 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T14:04:41.053 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T14:04:41.054 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:04:41.074 INFO:teuthology.orchestra.console:Power off smithi181 2023-07-27T14:04:41.075 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T14:04:41.086 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T14:04:41.098 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T14:04:41.098 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:04:41.123 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T14:04:41.188 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2023-07-27T14:04:41.225 INFO:teuthology.orchestra.console:Power off for smithi181 completed 2023-07-27T14:04:41.353 INFO:teuthology.lock.ops:unlocked: smithi142.front.sepia.ceph.com 2023-07-27T14:04:41.368 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com