2023-10-16T23:37:30.712 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-16T23:37:30.714 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:37:30.852 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1c29d35670>, signals=[15]) 2023-10-16T23:37:30.853 INFO:teuthology.lock.ops:Start node 'smithi132.front.sepia.ceph.com' reimaging 2023-10-16T23:37:30.854 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: reset os type and version on server 2023-10-16T23:37:30.854 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-10-16T23:37:30.876 INFO:teuthology.lock.ops:Start node 'smithi157.front.sepia.ceph.com' reimaging 2023-10-16T23:37:30.876 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: reset os type and version on server 2023-10-16T23:37:30.876 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-10-16T23:37:30.878 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /host {"name": "smithi132"} 2023-10-16T23:37:30.901 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /host {"name": "smithi157"} 2023-10-16T23:37:30.953 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-10-16T23:37:30.977 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-10-16T23:37:30.997 DEBUG:teuthology.provision.fog.smithi132:Requesting image smithi_centos_9.stream (ID 10) 2023-10-16T23:37:30.997 DEBUG:teuthology.provision.fog.smithi132:FOG request: PUT /host/237 {"imageID": 10} 2023-10-16T23:37:31.022 DEBUG:teuthology.provision.fog.smithi157:Requesting image smithi_centos_9.stream (ID 10) 2023-10-16T23:37:31.022 DEBUG:teuthology.provision.fog.smithi157:FOG request: PUT /host/262 {"imageID": 10} 2023-10-16T23:37:31.056 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of centos 9.stream 2023-10-16T23:37:31.056 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:37:31.073 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of centos 9.stream 2023-10-16T23:37:31.073 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:37:31.102 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /tasktype {"name": "deploy"} 2023-10-16T23:37:31.116 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /tasktype {"name": "deploy"} 2023-10-16T23:37:31.134 DEBUG:teuthology.provision.fog.smithi132:FOG request: POST /host/237/task {"taskTypeID": 1} 2023-10-16T23:37:31.147 DEBUG:teuthology.provision.fog.smithi157:FOG request: POST /host/262/task {"taskTypeID": 1} 2023-10-16T23:37:31.184 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:37:31.194 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:37:31.262 INFO:teuthology.orchestra.console.smithi132:Power off 2023-10-16T23:37:31.263 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T23:37:31.275 INFO:teuthology.orchestra.console.smithi157:Power off 2023-10-16T23:37:31.276 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T23:37:31.285 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2023-10-16T23:37:31.286 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:31.296 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-10-16T23:37:31.297 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:31.307 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-10-16T23:37:31.318 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-16T23:37:35.310 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:35.318 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:35.419 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-10-16T23:37:35.426 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-16T23:37:39.422 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:39.429 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:39.532 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-10-16T23:37:39.538 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-16T23:37:39.633 INFO:teuthology.orchestra.console.smithi132:Power off completed 2023-10-16T23:37:39.639 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-10-16T23:37:39.735 INFO:teuthology.orchestra.console.smithi132:Power on 2023-10-16T23:37:39.735 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-16T23:37:39.743 INFO:teuthology.orchestra.console.smithi157:Power on 2023-10-16T23:37:39.743 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-16T23:37:39.763 DEBUG:teuthology.orchestra.console.smithi157:power on output: Chassis Power Control: Up/On 2023-10-16T23:37:39.764 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:39.770 DEBUG:teuthology.orchestra.console.smithi132:power on output: Chassis Power Control: Up/On 2023-10-16T23:37:39.771 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:39.784 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-16T23:37:39.791 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-10-16T23:37:43.788 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:43.795 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:43.898 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-16T23:37:43.904 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-10-16T23:37:47.902 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:47.909 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T23:37:48.011 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-16T23:37:48.019 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-10-16T23:37:48.113 INFO:teuthology.orchestra.console.smithi157:Power on completed 2023-10-16T23:37:48.120 INFO:teuthology.orchestra.console.smithi132:Power on completed 2023-10-16T23:37:48.214 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish 2023-10-16T23:37:48.215 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:37:48.221 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2023-10-16T23:37:48.221 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:38:03.286 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:38:03.288 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:38:18.356 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:38:18.358 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:38:33.430 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:38:33.432 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:38:48.494 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:38:48.496 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:39:03.570 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:39:03.572 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:39:18.650 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:39:18.652 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:39:33.724 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:39:33.726 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:39:48.794 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:39:48.796 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:40:03.866 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:40:03.868 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:40:18.936 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:40:18.937 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-10-16T23:40:18.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:40:34.013 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-16T23:40:34.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:40:34.581 ERROR:teuthology.orchestra.connection:Error authenticating with smithi157.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-16T23:40:37.370 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-10-16T23:40:43.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:40:46.426 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-10-16T23:40:52.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:40:55.482 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-10-16T23:41:01.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:41:04.542 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-10-16T23:41:10.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:41:38.586 WARNING:teuthology.provision.fog.smithi157:timed out 2023-10-16T23:41:44.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:41:47.646 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-16T23:41:53.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:42:10.546 WARNING:teuthology.provision.fog.smithi132:timed out 2023-10-16T23:42:16.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:42:16.858 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-16T23:42:16.859 DEBUG:teuthology.orchestra.run.smithi132:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-16T23:42:17.202 INFO:teuthology.provision.fog.smithi132:Node is ready 2023-10-16T23:42:17.203 DEBUG:teuthology.orchestra.run.smithi132:> hostname 2023-10-16T23:42:17.218 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com 2023-10-16T23:42:17.218 DEBUG:teuthology.orchestra.run.smithi132:> grep smithi132.front.sepia.ceph.com /etc/hosts 2023-10-16T23:42:17.273 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132 2023-10-16T23:42:17.274 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-10-16T23:42:17.378 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132.front.sepia.ceph.com/smithi132/g' /etc/hosts 2023-10-16T23:42:17.406 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/172.21.15.132/172.21.15.132/g' /etc/hosts 2023-10-16T23:42:17.473 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132.front.sepia.ceph.com/smithi132/g' /etc/hostname 2023-10-16T23:42:17.540 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-10-16T23:42:17.606 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-10-16T23:42:17.660 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2023-10-16T23:42:17.660 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2023-10-16T23:42:17.660 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2023-10-16T23:42:17.661 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2023-10-16T23:42:17.661 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2023-10-16T23:42:17.661 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2023-10-16T23:42:17.661 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-16T23:42:17.661 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-10-16T23:42:17.662 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2023-10-16T23:42:17.662 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-16T23:42:17.662 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2023-10-16T23:42:17.662 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T23:42:17.662 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-16T23:42:17.663 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-16T23:42:17.663 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2023-10-16T23:42:17.664 DEBUG:teuthology.parallel:result is None 2023-10-16T23:42:53.650 WARNING:teuthology.provision.fog.smithi157:timed out 2023-10-16T23:42:59.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:42:59.834 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-16T23:42:59.834 DEBUG:teuthology.orchestra.run.smithi157:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-16T23:43:00.134 INFO:teuthology.provision.fog.smithi157:Node is ready 2023-10-16T23:43:00.135 DEBUG:teuthology.orchestra.run.smithi157:> hostname 2023-10-16T23:43:00.148 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com 2023-10-16T23:43:00.149 DEBUG:teuthology.orchestra.run.smithi157:> grep smithi157.front.sepia.ceph.com /etc/hosts 2023-10-16T23:43:00.204 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157 2023-10-16T23:43:00.205 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157 2023-10-16T23:43:00.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hosts 2023-10-16T23:43:00.351 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/172.21.15.157/172.21.15.157/g' /etc/hosts 2023-10-16T23:43:00.418 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hostname 2023-10-16T23:43:00.487 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157 2023-10-16T23:43:00.553 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-10-16T23:43:00.608 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2023-10-16T23:43:00.609 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2023-10-16T23:43:00.609 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2023-10-16T23:43:00.609 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2023-10-16T23:43:00.610 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2023-10-16T23:43:00.610 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2023-10-16T23:43:00.610 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-16T23:43:00.610 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-10-16T23:43:00.610 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2023-10-16T23:43:00.611 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-16T23:43:00.611 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2023-10-16T23:43:00.611 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T23:43:00.611 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-16T23:43:00.611 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-16T23:43:00.612 INFO:teuthology.provision.fog.smithi157:Deploy complete! 2023-10-16T23:43:00.612 DEBUG:teuthology.parallel:result is None 2023-10-16T23:43:00.779 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com 2023-10-16T23:43:00.780 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-16T23:43:00.810 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com 2023-10-16T23:43:00.811 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-16T23:43:00.828 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server 2023-10-16T23:43:00.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-16T23:43:00.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:43:00.913 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-10-16T23:43:00.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-16T23:43:00.982 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-10-16T23:43:01.037 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-10-16T23:43:01.038 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-10-16T23:43:01.093 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2023-10-16T23:43:01.093 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2023-10-16T23:43:01.093 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2023-10-16T23:43:01.094 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2023-10-16T23:43:01.094 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2023-10-16T23:43:01.094 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2023-10-16T23:43:01.094 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-16T23:43:01.094 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-10-16T23:43:01.094 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2023-10-16T23:43:01.095 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-16T23:43:01.095 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2023-10-16T23:43:01.095 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T23:43:01.095 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-16T23:43:01.095 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-16T23:43:01.096 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-10-16T23:43:01.119 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server 2023-10-16T23:43:01.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T23:43:01.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T23:43:01.202 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-10-16T23:43:01.226 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-16T23:43:01.226 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-10-16T23:43:01.280 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-10-16T23:43:01.281 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-10-16T23:43:01.334 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2023-10-16T23:43:01.334 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2023-10-16T23:43:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2023-10-16T23:43:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2023-10-16T23:43:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2023-10-16T23:43:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2023-10-16T23:43:01.335 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-16T23:43:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-10-16T23:43:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2023-10-16T23:43:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-16T23:43:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2023-10-16T23:43:01.336 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T23:43:01.337 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-16T23:43:01.337 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-16T23:43:01.337 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-10-16T23:43:01.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:43:01.446 INFO:teuthology.dispatcher.supervisor:Running job 7430692 2023-10-16T23:43:01.446 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner scheduled_sjust@teuthology --archive /home/teuthworker/archive/sjust-2023-10-16_23:13:53-crimson-rados-wip-crimson-scrub-testing-2023-10-15-distro-default-smithi/7430692 --name sjust-2023-10-16_23:13:53-crimson-rados-wip-crimson-scrub-testing-2023-10-15-distro-default-smithi --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/readwrite} -- /home/teuthworker/archive/sjust-2023-10-16_23:13:53-crimson-rados-wip-crimson-scrub-testing-2023-10-15-distro-default-smithi/7430692/orig.config.yaml 2023-10-16T23:43:01.449 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sjust-2023-10-16_23:13:53-crimson-rados-wip-crimson-scrub-testing-2023-10-15-distro-default-smithi/7430692 2023-10-16T23:43:01.450 INFO:teuthology.dispatcher.supervisor:Job PID: 2584686 2023-10-16T23:43:01.450 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-16T23:45:01.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:47:01.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:49:01.597 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:51:01.664 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:53:01.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:55:01.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:57:01.862 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T23:59:01.920 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T00:01:01.992 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T00:03:02.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T00:05:02.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T00:07:02.187 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T00:09:02.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T00:11:02.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T00:11:02.387 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-17T00:11:02.472 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-17T00:11:02.473 INFO:teuthology.nuke:Checking targets against current locks 2023-10-17T00:11:02.522 DEBUG:teuthology.nuke:shortname: smithi132 2023-10-17T00:11:02.526 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-17T00:11:02.547 DEBUG:teuthology.nuke:shortname: smithi157 2023-10-17T00:11:02.548 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-17T00:11:02.551 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2023-10-16_23:13:53-crimson-rados-wip-crimson-scrub-testing-2023-10-15-distro-default-smithi/7430692', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 23:37:29.943267', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGlLRVoTi0D5rjn0yRdkMVJs3F0lX2pXaLdxdjw8JcDWAxKZ+8qh72rdiSZQt5MmEUh6onprMlOHCLm90BXardM='} 2023-10-17T00:11:02.576 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2023-10-16_23:13:53-crimson-rados-wip-crimson-scrub-testing-2023-10-15-distro-default-smithi/7430692', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 23:37:29.944652', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEtcoN4GJ/htY/sNL2DHUKrzHdiJ9gR1omOxNRAFbaD6kEFdJbzg0vGMMEPnwCdJD28pwhAVZCsiWugNp4ddQDc='} 2023-10-17T00:11:02.606 INFO:teuthology.orchestra.console.smithi132:Power off 2023-10-17T00:11:02.607 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-17T00:11:02.628 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2023-10-17T00:11:02.629 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T00:11:02.643 INFO:teuthology.orchestra.console.smithi157:Power off 2023-10-17T00:11:02.644 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-17T00:11:02.652 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-10-17T00:11:02.664 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-10-17T00:11:02.665 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T00:11:02.684 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-17T00:11:02.754 INFO:teuthology.orchestra.console.smithi132:Power off completed 2023-10-17T00:11:02.785 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-10-17T00:11:02.904 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com 2023-10-17T00:11:02.928 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com