2023-09-23T16:22:57.596 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-23T16:22:57.597 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:22:57.735 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc8d1bc2670>, signals=[15]) 2023-09-23T16:22:57.735 INFO:teuthology.lock.ops:Start node 'smithi096.front.sepia.ceph.com' reimaging 2023-09-23T16:22:57.735 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: reset os type and version on server 2023-09-23T16:22:57.736 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2023-09-23T16:22:57.753 INFO:teuthology.lock.ops:Start node 'smithi204.front.sepia.ceph.com' reimaging 2023-09-23T16:22:57.753 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: reset os type and version on server 2023-09-23T16:22:57.753 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2023-09-23T16:22:57.754 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /host {"name": "smithi096"} 2023-09-23T16:22:57.778 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /host {"name": "smithi204"} 2023-09-23T16:22:57.824 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-23T16:22:57.855 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-23T16:22:57.862 DEBUG:teuthology.provision.fog.smithi096:Requesting image smithi_centos_8.stream (ID 3) 2023-09-23T16:22:57.862 DEBUG:teuthology.provision.fog.smithi096:FOG request: PUT /host/201 {"imageID": 3} 2023-09-23T16:22:57.896 DEBUG:teuthology.provision.fog.smithi204:Requesting image smithi_centos_8.stream (ID 3) 2023-09-23T16:22:57.896 DEBUG:teuthology.provision.fog.smithi204:FOG request: PUT /host/308 {"imageID": 3} 2023-09-23T16:22:57.913 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of centos 8.stream 2023-09-23T16:22:57.913 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:22:57.945 INFO:teuthology.provision.fog.smithi204:Scheduling deploy of centos 8.stream 2023-09-23T16:22:57.945 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:22:57.970 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /tasktype {"name": "deploy"} 2023-09-23T16:22:57.991 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /tasktype {"name": "deploy"} 2023-09-23T16:22:58.011 DEBUG:teuthology.provision.fog.smithi096:FOG request: POST /host/201/task {"taskTypeID": 1} 2023-09-23T16:22:58.022 DEBUG:teuthology.provision.fog.smithi204:FOG request: POST /host/308/task {"taskTypeID": 1} 2023-09-23T16:22:58.067 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:22:58.070 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:22:58.156 INFO:teuthology.orchestra.console.smithi096:Power off 2023-09-23T16:22:58.156 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T16:22:58.168 INFO:teuthology.orchestra.console.smithi204:Power off 2023-09-23T16:22:58.169 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T16:22:58.176 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off 2023-09-23T16:22:58.177 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T16:22:58.190 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2023-09-23T16:22:58.190 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T16:22:58.197 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2023-09-23T16:22:58.210 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-09-23T16:22:58.298 INFO:teuthology.orchestra.console.smithi096:Power off completed 2023-09-23T16:22:58.311 INFO:teuthology.orchestra.console.smithi204:Power off completed 2023-09-23T16:22:58.399 INFO:teuthology.orchestra.console.smithi096:Power on 2023-09-23T16:22:58.399 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-23T16:22:58.411 INFO:teuthology.orchestra.console.smithi204:Power on 2023-09-23T16:22:58.412 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-23T16:22:58.420 DEBUG:teuthology.orchestra.console.smithi096:power on output: Chassis Power Control: Up/On 2023-09-23T16:22:58.420 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T16:22:58.432 DEBUG:teuthology.orchestra.console.smithi204:power on output: Chassis Power Control: Up/On 2023-09-23T16:22:58.432 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T16:22:58.439 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2023-09-23T16:22:58.451 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-09-23T16:23:02.444 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T16:23:02.452 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T16:23:02.553 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2023-09-23T16:23:02.558 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-09-23T16:23:06.554 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T16:23:06.560 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T16:23:06.663 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2023-09-23T16:23:06.669 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2023-09-23T16:23:06.764 INFO:teuthology.orchestra.console.smithi096:Power on completed 2023-09-23T16:23:06.770 INFO:teuthology.orchestra.console.smithi204:Power on completed 2023-09-23T16:23:06.865 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish 2023-09-23T16:23:06.865 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:23:06.871 INFO:teuthology.provision.fog.smithi204:Waiting for deploy to finish 2023-09-23T16:23:06.871 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:23:21.946 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:23:21.947 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:23:37.023 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:23:37.024 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:23:52.094 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:23:52.095 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:24:07.178 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:24:07.179 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:24:22.254 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:24:22.255 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:24:37.326 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:24:37.327 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:24:52.401 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:24:52.403 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:25:07.474 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:25:07.475 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:25:22.546 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:25:22.547 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:25:37.614 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None 2023-09-23T16:25:37.615 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:25:37.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:25:52.684 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-09-23T16:25:52.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:25:56.090 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96 2023-09-23T16:26:02.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:05.178 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96 2023-09-23T16:26:11.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:11.194 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-09-23T16:26:14.266 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96 2023-09-23T16:26:17.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:20.250 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-09-23T16:26:20.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:23.322 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96 2023-09-23T16:26:26.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:29.306 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-09-23T16:26:29.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:32.378 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96 2023-09-23T16:26:35.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:38.363 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-09-23T16:26:38.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:44.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:26:47.450 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-09-23T16:26:53.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:27:38.414 WARNING:teuthology.provision.fog.smithi096:timed out 2023-09-23T16:27:44.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:27:44.641 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-23T16:27:44.642 DEBUG:teuthology.orchestra.run.smithi096:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-23T16:27:44.919 INFO:teuthology.provision.fog.smithi096:Node is ready 2023-09-23T16:27:44.919 DEBUG:teuthology.orchestra.run.smithi096:> hostname 2023-09-23T16:27:44.939 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com 2023-09-23T16:27:44.939 DEBUG:teuthology.orchestra.run.smithi096:> grep smithi096.front.sepia.ceph.com /etc/hosts 2023-09-23T16:27:44.998 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096 2023-09-23T16:27:44.998 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname smithi096 2023-09-23T16:27:45.103 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hosts 2023-09-23T16:27:45.132 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/172.21.15.96/172.21.15.96/g' /etc/hosts 2023-09-23T16:27:45.201 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hostname 2023-09-23T16:27:45.270 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname smithi096 2023-09-23T16:27:45.338 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2023-09-23T16:27:45.396 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2023-09-23T16:27:45.396 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8" 2023-09-23T16:27:45.396 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2023-09-23T16:27:45.396 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T16:27:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-23T16:27:45.398 INFO:teuthology.provision.fog.smithi096:Deploy complete! 2023-09-23T16:27:45.399 DEBUG:teuthology.parallel:result is None 2023-09-23T16:27:53.465 WARNING:teuthology.provision.fog.smithi204:timed out 2023-09-23T16:27:59.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:27:59.708 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-23T16:27:59.708 DEBUG:teuthology.orchestra.run.smithi204:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-23T16:27:59.993 INFO:teuthology.provision.fog.smithi204:Node is ready 2023-09-23T16:27:59.993 DEBUG:teuthology.orchestra.run.smithi204:> hostname 2023-09-23T16:28:00.012 INFO:teuthology.orchestra.run.smithi204.stdout:smithi204.front.sepia.ceph.com 2023-09-23T16:28:00.012 DEBUG:teuthology.orchestra.run.smithi204:> grep smithi204.front.sepia.ceph.com /etc/hosts 2023-09-23T16:28:00.071 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204 2023-09-23T16:28:00.072 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname smithi204 2023-09-23T16:28:00.172 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/smithi204.front.sepia.ceph.com/smithi204/g' /etc/hosts 2023-09-23T16:28:00.202 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/172.21.15.204/172.21.15.204/g' /etc/hosts 2023-09-23T16:28:00.270 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/smithi204.front.sepia.ceph.com/smithi204/g' /etc/hostname 2023-09-23T16:28:00.338 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname smithi204 2023-09-23T16:28:00.404 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2023-09-23T16:28:00.463 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2023-09-23T16:28:00.463 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8" 2023-09-23T16:28:00.463 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2023-09-23T16:28:00.463 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2023-09-23T16:28:00.464 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8" 2023-09-23T16:28:00.464 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8" 2023-09-23T16:28:00.464 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-23T16:28:00.464 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2023-09-23T16:28:00.464 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-23T16:28:00.464 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2023-09-23T16:28:00.464 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T16:28:00.465 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T16:28:00.465 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-23T16:28:00.466 INFO:teuthology.provision.fog.smithi204:Deploy complete! 2023-09-23T16:28:00.466 DEBUG:teuthology.parallel:result is None 2023-09-23T16:28:00.607 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com 2023-09-23T16:28:00.608 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-23T16:28:00.630 INFO:teuthology.lock.ops:Checking smithi204.front.sepia.ceph.com 2023-09-23T16:28:00.631 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-23T16:28:00.648 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server 2023-09-23T16:28:00.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2023-09-23T16:28:00.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:28:00.715 DEBUG:teuthology.orchestra.run.smithi096:> true 2023-09-23T16:28:00.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2023-09-23T16:28:00.844 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2023-09-23T16:28:00.902 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2023-09-23T16:28:00.903 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2023-09-23T16:28:00.960 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2023-09-23T16:28:00.960 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8" 2023-09-23T16:28:00.960 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2023-09-23T16:28:00.960 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T16:28:00.961 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-23T16:28:00.962 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2023-09-23T16:28:00.981 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: set os type and version on server 2023-09-23T16:28:00.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2023-09-23T16:28:00.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T16:28:01.047 DEBUG:teuthology.orchestra.run.smithi204:> true 2023-09-23T16:28:01.071 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2023-09-23T16:28:01.071 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2023-09-23T16:28:01.129 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2023-09-23T16:28:01.129 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2023-09-23T16:28:01.187 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2023-09-23T16:28:01.187 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8" 2023-09-23T16:28:01.187 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2023-09-23T16:28:01.187 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2023-09-23T16:28:01.187 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8" 2023-09-23T16:28:01.187 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8" 2023-09-23T16:28:01.187 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-23T16:28:01.187 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2023-09-23T16:28:01.188 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-23T16:28:01.188 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2023-09-23T16:28:01.188 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T16:28:01.188 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T16:28:01.188 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-23T16:28:01.188 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2023-09-23T16:28:01.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:28:01.292 INFO:teuthology.dispatcher.supervisor:Running job 7401739 2023-09-23T16:28:01.292 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401739 --name teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi --description upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} -- /home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401739/orig.config.yaml 2023-09-23T16:28:01.295 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401739 2023-09-23T16:28:01.295 INFO:teuthology.dispatcher.supervisor:Job PID: 3468132 2023-09-23T16:28:01.295 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-23T16:30:01.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:32:01.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:34:01.723 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:36:01.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:38:02.078 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:40:02.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:42:02.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:44:02.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:46:02.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:48:02.902 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:50:03.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:52:03.227 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:54:03.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:56:03.500 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T16:58:03.656 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:00:03.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:02:03.992 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:04:04.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:06:04.317 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:08:04.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:10:04.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:12:04.800 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:14:04.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:16:05.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:18:05.129 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:20:05.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:22:05.481 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:24:05.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:26:05.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:28:05.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:30:06.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:32:06.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:34:06.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:36:06.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:38:06.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:40:06.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T17:40:06.972 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-09-23T17:40:07.173 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-23T17:40:07.173 INFO:teuthology.nuke:Checking targets against current locks 2023-09-23T17:40:07.234 DEBUG:teuthology.nuke:shortname: smithi096 2023-09-23T17:40:07.234 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T17:40:07.261 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401739', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 16:22:56.825864', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNjaKnK+6FnL4+psv0IJ7Oihg/BzjUt8sYx6prdqcrlNpsA7Ki69rUzhMelk/TO4Aw7Gjd237Loraz/BJVGRwL0='} 2023-09-23T17:40:07.262 DEBUG:teuthology.nuke:shortname: smithi204 2023-09-23T17:40:07.262 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T17:40:07.285 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401739', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 16:22:56.824016', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBILShYE0lvM8tYIzst4nxC11/62kP3da/N2bo9DQqBWudb3ZoyYb0zwdDOjQQg7Dmwn7KRvoY2DnfnRN+HlsF8o='} 2023-09-23T17:40:07.313 INFO:teuthology.orchestra.console.smithi096:Power off 2023-09-23T17:40:07.313 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T17:40:07.334 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off 2023-09-23T17:40:07.334 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T17:40:07.351 INFO:teuthology.orchestra.console.smithi204:Power off 2023-09-23T17:40:07.351 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T17:40:07.359 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2023-09-23T17:40:07.372 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2023-09-23T17:40:07.372 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T17:40:07.391 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-09-23T17:40:07.460 INFO:teuthology.orchestra.console.smithi096:Power off completed 2023-09-23T17:40:07.492 INFO:teuthology.orchestra.console.smithi204:Power off completed 2023-09-23T17:40:07.613 INFO:teuthology.lock.ops:unlocked: smithi096.front.sepia.ceph.com 2023-09-23T17:40:07.638 INFO:teuthology.lock.ops:unlocked: smithi204.front.sepia.ceph.com