2023-07-03T08:08:46.251 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-03T08:08:46.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:08:46.253 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:08:46.416 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb5defc3790>, signals=[15]) 2023-07-03T08:08:46.418 INFO:teuthology.lock.ops:Start node 'smithi145.front.sepia.ceph.com' reimaging 2023-07-03T08:08:46.418 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: reset os type and version on server 2023-07-03T08:08:46.418 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2023-07-03T08:08:46.443 INFO:teuthology.lock.ops:Start node 'smithi160.front.sepia.ceph.com' reimaging 2023-07-03T08:08:46.444 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: reset os type and version on server 2023-07-03T08:08:46.444 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2023-07-03T08:08:46.446 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /host {"name": "smithi145"} 2023-07-03T08:08:46.468 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /host {"name": "smithi160"} 2023-07-03T08:08:46.520 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-03T08:08:46.539 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-03T08:08:46.560 DEBUG:teuthology.provision.fog.smithi145:Requesting image smithi_centos_8.stream (ID 3) 2023-07-03T08:08:46.560 DEBUG:teuthology.provision.fog.smithi145:FOG request: PUT /host/250 {"imageID": 3} 2023-07-03T08:08:46.585 DEBUG:teuthology.provision.fog.smithi160:Requesting image smithi_centos_8.stream (ID 3) 2023-07-03T08:08:46.586 DEBUG:teuthology.provision.fog.smithi160:FOG request: PUT /host/265 {"imageID": 3} 2023-07-03T08:08:46.602 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of centos 8.stream 2023-07-03T08:08:46.603 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:08:46.624 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of centos 8.stream 2023-07-03T08:08:46.625 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:08:46.644 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /tasktype {"name": "deploy"} 2023-07-03T08:08:46.671 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /tasktype {"name": "deploy"} 2023-07-03T08:08:46.679 DEBUG:teuthology.provision.fog.smithi145:FOG request: POST /host/250/task {"taskTypeID": 1} 2023-07-03T08:08:46.709 DEBUG:teuthology.provision.fog.smithi160:FOG request: POST /host/265/task {"taskTypeID": 1} 2023-07-03T08:08:46.731 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:08:46.758 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:08:46.813 INFO:teuthology.orchestra.console:Power off smithi145 2023-07-03T08:08:46.814 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-03T08:08:46.839 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:08:46.856 INFO:teuthology.orchestra.console:Power off smithi160 2023-07-03T08:08:46.857 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-03T08:08:46.878 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:08:46.967 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2023-07-03T08:08:47.000 INFO:teuthology.orchestra.console:Power off for smithi160 completed 2023-07-03T08:08:47.069 INFO:teuthology.orchestra.console:Power on smithi145 2023-07-03T08:08:47.069 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-03T08:08:47.092 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:08:47.101 INFO:teuthology.orchestra.console:Power on smithi160 2023-07-03T08:08:47.102 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-03T08:08:47.126 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:08:51.116 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:08:51.148 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:08:55.228 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:08:55.261 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:08:55.441 INFO:teuthology.orchestra.console:Power on for smithi145 completed 2023-07-03T08:08:55.473 INFO:teuthology.orchestra.console:Power on for smithi160 completed 2023-07-03T08:08:55.543 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2023-07-03T08:08:55.544 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:08:55.574 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish 2023-07-03T08:08:55.574 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:09:10.600 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:09:10.628 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:09:25.669 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:09:25.679 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:09:40.732 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:09:40.735 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:09:55.800 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:09:55.801 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:10:10.864 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:10:10.865 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:10:25.932 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:10:25.934 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:10:40.997 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:10:41.000 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:10:56.060 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:10:56.061 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:11:11.132 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:11:11.133 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:11:26.208 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:11:26.210 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:11:41.261 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-07-03T08:11:41.262 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-03T08:11:41.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:11:41.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:11:44.368 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-03T08:11:44.369 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-03T08:11:50.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:11:50.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:11:53.424 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-03T08:11:53.425 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-03T08:11:59.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:11:59.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:02.483 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-03T08:12:02.484 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-03T08:12:08.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:08.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:11.572 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-03T08:12:11.573 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-03T08:12:17.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:17.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:20.656 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-03T08:12:20.657 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-03T08:12:26.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:26.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:29.712 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-03T08:12:29.713 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-03T08:12:35.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:35.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:38.800 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-03T08:12:44.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:12:47.856 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-03T08:12:53.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:13:35.752 WARNING:teuthology.provision.fog.smithi145:timed out 2023-07-03T08:13:41.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:13:42.028 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-03T08:13:42.029 DEBUG:teuthology.orchestra.run.smithi145:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-03T08:13:42.303 INFO:teuthology.provision.fog.smithi145:Node is ready 2023-07-03T08:13:42.304 DEBUG:teuthology.orchestra.run.smithi145:> hostname 2023-07-03T08:13:42.324 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2023-07-03T08:13:42.324 DEBUG:teuthology.orchestra.run.smithi145:> grep smithi145.front.sepia.ceph.com /etc/hosts 2023-07-03T08:13:42.384 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2023-07-03T08:13:42.384 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname smithi145 2023-07-03T08:13:42.491 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/smithi145.front.sepia.ceph.com/smithi145/g' /etc/hosts 2023-07-03T08:13:42.521 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/172.21.15.145/172.21.15.145/g' /etc/hosts 2023-07-03T08:13:42.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/smithi145.front.sepia.ceph.com/smithi145/g' /etc/hostname 2023-07-03T08:13:42.657 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname smithi145 2023-07-03T08:13:42.724 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2023-07-03T08:13:42.782 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2023-07-03T08:13:42.783 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8" 2023-07-03T08:13:42.783 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2023-07-03T08:13:42.783 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2023-07-03T08:13:42.783 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8" 2023-07-03T08:13:42.783 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2023-07-03T08:13:42.783 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-03T08:13:42.783 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2023-07-03T08:13:42.784 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-03T08:13:42.784 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2023-07-03T08:13:42.784 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-03T08:13:42.784 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-03T08:13:42.784 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-03T08:13:42.785 INFO:teuthology.provision.fog.smithi145:Deploy complete! 2023-07-03T08:13:42.786 DEBUG:teuthology.parallel:result is None 2023-07-03T08:13:53.872 WARNING:teuthology.provision.fog.smithi160:timed out 2023-07-03T08:13:59.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:14:00.120 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-03T08:14:00.120 DEBUG:teuthology.orchestra.run.smithi160:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-03T08:14:00.373 INFO:teuthology.provision.fog.smithi160:Node is ready 2023-07-03T08:14:00.373 DEBUG:teuthology.orchestra.run.smithi160:> hostname 2023-07-03T08:14:00.393 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com 2023-07-03T08:14:00.393 DEBUG:teuthology.orchestra.run.smithi160:> grep smithi160.front.sepia.ceph.com /etc/hosts 2023-07-03T08:14:00.453 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160 2023-07-03T08:14:00.454 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2023-07-03T08:14:00.553 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hosts 2023-07-03T08:14:00.584 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/172.21.15.160/172.21.15.160/g' /etc/hosts 2023-07-03T08:14:00.652 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hostname 2023-07-03T08:14:00.721 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2023-07-03T08:14:00.789 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2023-07-03T08:14:00.846 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2023-07-03T08:14:00.847 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8" 2023-07-03T08:14:00.847 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2023-07-03T08:14:00.848 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2023-07-03T08:14:00.848 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8" 2023-07-03T08:14:00.848 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2023-07-03T08:14:00.848 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-03T08:14:00.848 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2023-07-03T08:14:00.848 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-03T08:14:00.848 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2023-07-03T08:14:00.849 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-03T08:14:00.849 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-03T08:14:00.849 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-03T08:14:00.849 INFO:teuthology.provision.fog.smithi160:Deploy complete! 2023-07-03T08:14:00.850 DEBUG:teuthology.parallel:result is None 2023-07-03T08:14:01.008 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com 2023-07-03T08:14:01.009 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-03T08:14:01.046 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com 2023-07-03T08:14:01.047 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-03T08:14:01.073 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server 2023-07-03T08:14:01.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2023-07-03T08:14:01.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:14:01.144 DEBUG:teuthology.orchestra.run.smithi145:> true 2023-07-03T08:14:01.234 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2023-07-03T08:14:01.235 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2023-07-03T08:14:01.293 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2023-07-03T08:14:01.294 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2023-07-03T08:14:01.351 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2023-07-03T08:14:01.352 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8" 2023-07-03T08:14:01.352 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2023-07-03T08:14:01.352 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2023-07-03T08:14:01.352 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8" 2023-07-03T08:14:01.353 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2023-07-03T08:14:01.353 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-03T08:14:01.353 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2023-07-03T08:14:01.353 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-03T08:14:01.353 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2023-07-03T08:14:01.353 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-03T08:14:01.353 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-03T08:14:01.354 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-03T08:14:01.354 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2023-07-03T08:14:01.379 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server 2023-07-03T08:14:01.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-03T08:14:01.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-03T08:14:01.446 DEBUG:teuthology.orchestra.run.smithi160:> true 2023-07-03T08:14:01.515 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-03T08:14:01.516 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2023-07-03T08:14:01.574 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2023-07-03T08:14:01.575 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2023-07-03T08:14:01.633 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2023-07-03T08:14:01.633 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8" 2023-07-03T08:14:01.633 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2023-07-03T08:14:01.633 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2023-07-03T08:14:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8" 2023-07-03T08:14:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2023-07-03T08:14:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-03T08:14:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2023-07-03T08:14:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-03T08:14:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2023-07-03T08:14:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-03T08:14:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-03T08:14:01.635 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-03T08:14:01.635 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2023-07-03T08:14:01.654 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:14:01.655 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:14:01.748 INFO:teuthology.dispatcher.supervisor:Running job 7324588 2023-07-03T08:14:01.748 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology -v --owner scheduled_mkogan@teuthology --archive /home/teuthworker/archive/mkogan-2023-07-03_07:51:01-rgw-wip-T52363-xrmeng8756-i001-distro-default-smithi/7324588 --name mkogan-2023-07-03_07:51:01-rgw-wip-T52363-xrmeng8756-i001-distro-default-smithi --description rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_8} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} -- /home/teuthworker/archive/mkogan-2023-07-03_07:51:01-rgw-wip-T52363-xrmeng8756-i001-distro-default-smithi/7324588/orig.config.yaml 2023-07-03T08:14:01.753 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mkogan-2023-07-03_07:51:01-rgw-wip-T52363-xrmeng8756-i001-distro-default-smithi/7324588 2023-07-03T08:14:01.754 INFO:teuthology.dispatcher.supervisor:Job PID: 13848 2023-07-03T08:14:01.755 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-03T08:16:01.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:16:01.851 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:18:01.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:18:01.989 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:20:02.166 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:20:02.167 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:22:02.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:22:02.343 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:24:02.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:24:02.503 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:26:02.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:26:02.678 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:28:02.853 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:28:02.854 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:30:03.029 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:30:03.029 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:32:03.202 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:32:03.203 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:34:03.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:34:03.386 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:36:03.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:36:03.572 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:38:03.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:38:03.754 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:40:03.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:40:03.934 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:42:04.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:42:04.118 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:44:04.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:44:04.287 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:46:04.460 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:46:04.461 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:48:04.635 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:48:04.636 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:50:04.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:50:04.806 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:52:04.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-03T08:52:04.977 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-03T08:52:05.045 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-03T08:52:05.222 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-03T08:52:05.222 INFO:teuthology.nuke:Checking targets against current locks 2023-07-03T08:52:05.274 DEBUG:teuthology.nuke:shortname: smithi145 2023-07-03T08:52:05.275 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-03T08:52:05.298 DEBUG:teuthology.nuke:shortname: smithi160 2023-07-03T08:52:05.299 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-03T08:52:05.301 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mkogan-2023-07-03_07:51:01-rgw-wip-T52363-xrmeng8756-i001-distro-default-smithi/7324588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-03 08:08:45.421011', 'locked_by': 'scheduled_mkogan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnHMc+EV3Bc20USH9MIrWXkcyZ6o7XiKoHZK7dXYXwx1pk8sY8K3q9T0qyzmpn+nnjnSxUs6V8UqFKhWMkYhsa63iMCbRxhUukWijQZUPDXDZmT29FMoK5zQ6Mddo8Lp1KU+gzVl9Q5pqD/HennAaU/RWkd9VMb1K5Tkl/MYaz7mVh0y8JoFgT/EGAWS6cf3lBCQFh3c1PmpUEx0IMDfuIvz0EhgzkHt8tIhR5YkBvo/aGLI1u5KiUDgy9nW9pn4f5Au8kJgIfyuNB080htMbkru61m4bjjIFQBsRy6oxwAf0MLdOkqEek7IsJNx/DWip23sF6iw4EtAPTypWCbDL48DmIaFIyTbcAOPUds+KuzJixFFKAeddz4V4fs9CFJGp0MTDem1mqT63EtziYuLzO18bJi3Yt9Rnc660wt0Eb95S94W6auMwFStipaFZpRgkNE1kE6iK7r+ZOlpIh/mlwGj0PxOAdgiAib5yOa0kDBQiqD/NpWLsD2Q0mk3W+mk8='} 2023-07-03T08:52:05.322 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mkogan-2023-07-03_07:51:01-rgw-wip-T52363-xrmeng8756-i001-distro-default-smithi/7324588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-03 08:08:45.422988', 'locked_by': 'scheduled_mkogan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9iBcSLBR5YsqOzu5j4Bb178+f9jPIKjXLIlDwBi9D5Ou/XsqJYpk2s1ZPWAwH/9h8wAq7Cp8IFjgi65ViexAg='} 2023-07-03T08:52:05.355 INFO:teuthology.orchestra.console:Power off smithi145 2023-07-03T08:52:05.356 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-03T08:52:05.377 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:52:05.391 INFO:teuthology.orchestra.console:Power off smithi160 2023-07-03T08:52:05.392 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-03T08:52:05.410 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-03T08:52:05.500 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2023-07-03T08:52:05.531 INFO:teuthology.orchestra.console:Power off for smithi160 completed 2023-07-03T08:52:05.660 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com 2023-07-03T08:52:05.677 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com