2023-09-13T22:22:41.891 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-13T22:22:41.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:22:42.033 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd98e389b80>, signals=[15]) 2023-09-13T22:22:42.034 INFO:teuthology.lock.ops:Start node 'smithi067.front.sepia.ceph.com' reimaging 2023-09-13T22:22:42.034 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: reset os type and version on server 2023-09-13T22:22:42.034 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2023-09-13T22:22:42.052 INFO:teuthology.lock.ops:Start node 'smithi172.front.sepia.ceph.com' reimaging 2023-09-13T22:22:42.053 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: reset os type and version on server 2023-09-13T22:22:42.053 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-09-13T22:22:42.054 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /host {"name": "smithi067"} 2023-09-13T22:22:42.077 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /host {"name": "smithi172"} 2023-09-13T22:22:42.135 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-13T22:22:42.147 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-13T22:22:42.191 DEBUG:teuthology.provision.fog.smithi067:Requesting image smithi_centos_8.stream (ID 3) 2023-09-13T22:22:42.191 DEBUG:teuthology.provision.fog.smithi067:FOG request: PUT /host/172 {"imageID": 3} 2023-09-13T22:22:42.196 DEBUG:teuthology.provision.fog.smithi172:Requesting image smithi_centos_8.stream (ID 3) 2023-09-13T22:22:42.196 DEBUG:teuthology.provision.fog.smithi172:FOG request: PUT /host/277 {"imageID": 3} 2023-09-13T22:22:42.242 INFO:teuthology.provision.fog.smithi067:Scheduling deploy of centos 8.stream 2023-09-13T22:22:42.242 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:22:42.245 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 8.stream 2023-09-13T22:22:42.245 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:22:42.315 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /tasktype {"name": "deploy"} 2023-09-13T22:22:42.318 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /tasktype {"name": "deploy"} 2023-09-13T22:22:42.350 DEBUG:teuthology.provision.fog.smithi172:FOG request: POST /host/277/task {"taskTypeID": 1} 2023-09-13T22:22:42.355 DEBUG:teuthology.provision.fog.smithi067:FOG request: POST /host/172/task {"taskTypeID": 1} 2023-09-13T22:22:42.416 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:22:42.419 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:22:42.517 INFO:teuthology.orchestra.console.smithi172:Power off 2023-09-13T22:22:42.518 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T22:22:42.529 INFO:teuthology.orchestra.console.smithi067:Power off 2023-09-13T22:22:42.529 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T22:22:42.539 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2023-09-13T22:22:42.540 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:42.550 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2023-09-13T22:22:42.550 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:42.562 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-09-13T22:22:42.572 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-09-13T22:22:46.567 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:46.573 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:46.676 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-09-13T22:22:46.681 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-09-13T22:22:50.678 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:50.686 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:50.788 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-09-13T22:22:50.796 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-09-13T22:22:54.790 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:54.797 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:54.900 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-09-13T22:22:54.906 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2023-09-13T22:22:55.002 INFO:teuthology.orchestra.console.smithi172:Power off completed 2023-09-13T22:22:55.007 INFO:teuthology.orchestra.console.smithi067:Power off completed 2023-09-13T22:22:55.103 INFO:teuthology.orchestra.console.smithi172:Power on 2023-09-13T22:22:55.103 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-13T22:22:55.114 INFO:teuthology.orchestra.console.smithi067:Power on 2023-09-13T22:22:55.114 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-13T22:22:55.126 DEBUG:teuthology.orchestra.console.smithi172:power on output: Chassis Power Control: Up/On 2023-09-13T22:22:55.127 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:55.134 DEBUG:teuthology.orchestra.console.smithi067:power on output: Chassis Power Control: Up/On 2023-09-13T22:22:55.135 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:55.146 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-09-13T22:22:55.155 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2023-09-13T22:22:59.150 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:59.157 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T22:22:59.259 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-09-13T22:22:59.265 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-09-13T22:22:59.361 INFO:teuthology.orchestra.console.smithi172:Power on completed 2023-09-13T22:22:59.367 INFO:teuthology.orchestra.console.smithi067:Power on completed 2023-09-13T22:22:59.462 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2023-09-13T22:22:59.463 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:22:59.467 INFO:teuthology.provision.fog.smithi067:Waiting for deploy to finish 2023-09-13T22:22:59.468 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:23:14.538 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:23:14.540 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:23:29.626 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:23:29.627 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:23:44.703 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:23:44.705 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:23:59.782 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:23:59.783 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:24:14.866 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:24:14.867 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:24:29.947 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:24:29.949 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:24:45.026 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:24:45.028 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:25:00.118 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:25:00.120 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:25:15.193 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:25:15.198 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:25:30.282 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:25:30.284 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:25:45.362 DEBUG:teuthology.provision.fog.smithi067:FOG request: GET /task/active None 2023-09-13T22:25:45.363 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:25:45.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:00.442 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-09-13T22:26:00.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:03.802 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-09-13T22:26:09.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:12.862 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-09-13T22:26:18.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:18.906 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-09-13T22:26:21.950 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-09-13T22:26:24.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:27.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:27.962 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-09-13T22:26:31.034 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-09-13T22:26:33.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:37.018 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-09-13T22:26:37.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:40.090 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-09-13T22:26:43.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:46.074 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-09-13T22:26:46.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:52.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:26:55.130 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-09-13T22:27:01.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:27:46.138 WARNING:teuthology.provision.fog.smithi067:timed out 2023-09-13T22:27:52.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:27:52.446 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-13T22:27:52.447 DEBUG:teuthology.orchestra.run.smithi067:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-13T22:27:52.708 INFO:teuthology.provision.fog.smithi067:Node is ready 2023-09-13T22:27:52.708 DEBUG:teuthology.orchestra.run.smithi067:> hostname 2023-09-13T22:27:52.729 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067.front.sepia.ceph.com 2023-09-13T22:27:52.730 DEBUG:teuthology.orchestra.run.smithi067:> grep smithi067.front.sepia.ceph.com /etc/hosts 2023-09-13T22:27:52.789 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67 smithi067.front.sepia.ceph.com smithi067 2023-09-13T22:27:52.789 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname smithi067 2023-09-13T22:27:52.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i -e 's/smithi067.front.sepia.ceph.com/smithi067/g' /etc/hosts 2023-09-13T22:27:52.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i -e 's/172.21.15.67/172.21.15.67/g' /etc/hosts 2023-09-13T22:27:52.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i -e 's/smithi067.front.sepia.ceph.com/smithi067/g' /etc/hostname 2023-09-13T22:27:53.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname smithi067 2023-09-13T22:27:53.121 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2023-09-13T22:27:53.180 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2023-09-13T22:27:53.180 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8" 2023-09-13T22:27:53.181 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2023-09-13T22:27:53.181 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2023-09-13T22:27:53.181 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8" 2023-09-13T22:27:53.181 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8" 2023-09-13T22:27:53.181 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-13T22:27:53.182 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2023-09-13T22:27:53.182 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-13T22:27:53.182 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2023-09-13T22:27:53.182 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T22:27:53.182 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T22:27:53.183 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-13T22:27:53.183 INFO:teuthology.provision.fog.smithi067:Deploy complete! 2023-09-13T22:27:53.327 DEBUG:teuthology.parallel:result is None 2023-09-13T22:28:01.146 WARNING:teuthology.provision.fog.smithi172:timed out 2023-09-13T22:28:07.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:28:07.363 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-13T22:28:07.363 DEBUG:teuthology.orchestra.run.smithi172:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-13T22:28:07.678 INFO:teuthology.provision.fog.smithi172:Node is ready 2023-09-13T22:28:07.679 DEBUG:teuthology.orchestra.run.smithi172:> hostname 2023-09-13T22:28:07.698 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com 2023-09-13T22:28:07.699 DEBUG:teuthology.orchestra.run.smithi172:> grep smithi172.front.sepia.ceph.com /etc/hosts 2023-09-13T22:28:07.758 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2023-09-13T22:28:07.759 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172 2023-09-13T22:28:07.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi172.front.sepia.ceph.com/smithi172/g' /etc/hosts 2023-09-13T22:28:07.894 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/172.21.15.172/172.21.15.172/g' /etc/hosts 2023-09-13T22:28:07.963 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi172.front.sepia.ceph.com/smithi172/g' /etc/hostname 2023-09-13T22:28:08.031 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172 2023-09-13T22:28:08.099 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-09-13T22:28:08.157 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2023-09-13T22:28:08.157 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8" 2023-09-13T22:28:08.157 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2023-09-13T22:28:08.157 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2023-09-13T22:28:08.158 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8" 2023-09-13T22:28:08.158 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2023-09-13T22:28:08.158 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-13T22:28:08.158 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-09-13T22:28:08.159 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-13T22:28:08.159 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2023-09-13T22:28:08.159 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T22:28:08.159 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T22:28:08.159 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-13T22:28:08.160 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2023-09-13T22:28:08.160 DEBUG:teuthology.parallel:result is None 2023-09-13T22:28:08.320 INFO:teuthology.lock.ops:Checking smithi067.front.sepia.ceph.com 2023-09-13T22:28:08.321 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-13T22:28:08.339 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2023-09-13T22:28:08.340 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-13T22:28:08.357 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: set os type and version on server 2023-09-13T22:28:08.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-09-13T22:28:08.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:28:08.425 DEBUG:teuthology.orchestra.run.smithi067:> true 2023-09-13T22:28:08.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-09-13T22:28:08.557 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2023-09-13T22:28:08.615 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2023-09-13T22:28:08.616 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2023-09-13T22:28:08.674 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2023-09-13T22:28:08.674 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8" 2023-09-13T22:28:08.675 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2023-09-13T22:28:08.675 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2023-09-13T22:28:08.675 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8" 2023-09-13T22:28:08.675 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8" 2023-09-13T22:28:08.675 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-13T22:28:08.676 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2023-09-13T22:28:08.676 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-13T22:28:08.676 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2023-09-13T22:28:08.676 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T22:28:08.677 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T22:28:08.677 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-13T22:28:08.677 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2023-09-13T22:28:08.699 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2023-09-13T22:28:08.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-09-13T22:28:08.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T22:28:08.765 DEBUG:teuthology.orchestra.run.smithi172:> true 2023-09-13T22:28:08.791 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-09-13T22:28:08.791 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2023-09-13T22:28:08.849 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2023-09-13T22:28:08.849 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-09-13T22:28:08.906 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2023-09-13T22:28:08.907 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8" 2023-09-13T22:28:08.907 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2023-09-13T22:28:08.907 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2023-09-13T22:28:08.907 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8" 2023-09-13T22:28:08.908 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2023-09-13T22:28:08.908 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-13T22:28:08.908 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-09-13T22:28:08.908 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-13T22:28:08.908 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2023-09-13T22:28:08.909 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T22:28:08.909 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T22:28:08.909 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-13T22:28:08.909 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-09-13T22:28:08.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:28:09.017 INFO:teuthology.dispatcher.supervisor:Running job 7396641 2023-09-13T22:28:09.018 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2023-09-13_20:47:39-rgw-wip-62506-reef-distro-default-smithi/7396641 --name cbodley-2023-09-13_20:47:39-rgw-wip-62506-reef-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi} -- /home/teuthworker/archive/cbodley-2023-09-13_20:47:39-rgw-wip-62506-reef-distro-default-smithi/7396641/orig.config.yaml 2023-09-13T22:28:09.021 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2023-09-13_20:47:39-rgw-wip-62506-reef-distro-default-smithi/7396641 2023-09-13T22:28:09.022 INFO:teuthology.dispatcher.supervisor:Job PID: 450410 2023-09-13T22:28:09.022 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-13T22:30:09.122 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:32:09.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:34:09.443 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:36:09.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:38:09.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:40:09.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:42:10.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:44:10.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:46:10.324 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:48:10.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:50:10.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:52:10.818 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:54:10.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:56:11.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T22:58:11.311 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:00:11.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:02:11.638 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:04:11.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:06:11.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:08:12.123 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:10:12.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:12:12.450 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:14:12.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:16:12.779 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:18:13.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:20:13.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:22:13.458 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:24:13.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:26:13.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:28:13.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:30:14.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:32:14.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:34:14.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:36:14.715 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:38:14.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:40:15.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:42:15.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:44:15.413 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:46:15.586 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:48:15.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:50:15.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:52:16.075 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:54:16.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:56:16.398 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:58:16.548 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:00:16.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:02:16.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:04:17.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:06:17.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:08:17.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:10:17.570 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:12:17.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:14:17.903 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:16:18.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:18:18.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:20:18.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:22:18.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:24:18.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:26:18.864 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:28:19.027 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:30:19.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:32:19.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:34:19.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:36:19.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:38:19.876 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-14T00:38:19.975 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-09-14T00:38:20.155 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-14T00:38:20.155 INFO:teuthology.nuke:Checking targets against current locks 2023-09-14T00:38:20.203 DEBUG:teuthology.nuke:shortname: smithi067 2023-09-14T00:38:20.203 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-14T00:38:20.225 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-09-13_20:47:39-rgw-wip-62506-reef-distro-default-smithi/7396641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 22:22:41.091045', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC8J2KvWdkzfhcEIKFBhe2kk3B8DqBVYGHmmFP1UekAwYsFhHhFfKN/mC/mmBWpSDHjEm/gSLT2n6T+BL5wtoi4='} 2023-09-14T00:38:20.228 DEBUG:teuthology.nuke:shortname: smithi172 2023-09-14T00:38:20.228 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-14T00:38:20.251 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-09-13_20:47:39-rgw-wip-62506-reef-distro-default-smithi/7396641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 22:22:41.089494', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIuxDT0NDgDEzLfkRi+WoSs6OodGAhEchWEgDg6gH9aHddqKkKP6IA1Ds+7eFYk27Mf/fVZk8ZpAtd1cosFXTcY='} 2023-09-14T00:38:20.271 INFO:teuthology.orchestra.console.smithi067:Power off 2023-09-14T00:38:20.272 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-14T00:38:20.291 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2023-09-14T00:38:20.291 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-14T00:38:20.307 INFO:teuthology.orchestra.console.smithi172:Power off 2023-09-14T00:38:20.307 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-14T00:38:20.315 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2023-09-14T00:38:20.327 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2023-09-14T00:38:20.328 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-14T00:38:20.350 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-09-14T00:38:20.417 INFO:teuthology.orchestra.console.smithi067:Power off completed 2023-09-14T00:38:20.451 INFO:teuthology.orchestra.console.smithi172:Power off completed 2023-09-14T00:38:20.566 INFO:teuthology.lock.ops:unlocked: smithi067.front.sepia.ceph.com 2023-09-14T00:38:20.598 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com