2023-12-24T17:25:14.268 INFO:root:teuthology version: 0.0.1.dev226+g9680d2a 2023-12-24T17:25:14.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:25:14.415 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6c1ff97940>, signals=[15]) 2023-12-24T17:25:14.416 INFO:teuthology.lock.ops:Start node 'smithi133.front.sepia.ceph.com' reimaging 2023-12-24T17:25:14.416 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: reset os type and version on server 2023-12-24T17:25:14.416 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-12-24T17:25:14.440 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging 2023-12-24T17:25:14.440 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server 2023-12-24T17:25:14.441 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-12-24T17:25:14.559 DEBUG:teuthology.provision.fog.smithi133:Requesting image smithi_centos_9.stream (ID 10) 2023-12-24T17:25:14.571 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_centos_9.stream (ID 10) 2023-12-24T17:25:14.607 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of centos 9.stream 2023-12-24T17:25:14.631 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 9.stream 2023-12-24T17:25:14.868 INFO:teuthology.orchestra.console.smithi133:Power off 2023-12-24T17:25:14.868 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-24T17:25:14.888 INFO:teuthology.orchestra.console.smithi181:Power off 2023-12-24T17:25:14.889 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-24T17:25:14.897 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2023-12-24T17:25:14.897 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:14.911 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-12-24T17:25:14.911 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:14.921 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-12-24T17:25:14.935 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-12-24T17:25:18.925 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:18.936 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:19.035 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-12-24T17:25:19.045 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-12-24T17:25:23.038 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:23.046 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:23.148 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-12-24T17:25:23.155 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-12-24T17:25:23.256 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-12-24T17:25:23.357 INFO:teuthology.orchestra.console.smithi181:Power on 2023-12-24T17:25:23.357 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-24T17:25:23.381 DEBUG:teuthology.orchestra.console.smithi181:power on output: Chassis Power Control: Up/On 2023-12-24T17:25:23.381 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:23.403 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-12-24T17:25:27.149 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:27.258 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2023-12-24T17:25:27.359 INFO:teuthology.orchestra.console.smithi133:Power off completed 2023-12-24T17:25:27.405 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:27.460 INFO:teuthology.orchestra.console.smithi133:Power on 2023-12-24T17:25:27.461 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-24T17:25:27.484 DEBUG:teuthology.orchestra.console.smithi133:power on output: Chassis Power Control: Up/On 2023-12-24T17:25:27.485 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:27.508 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2023-12-24T17:25:27.525 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-12-24T17:25:27.626 INFO:teuthology.orchestra.console.smithi181:Power on completed 2023-12-24T17:25:27.726 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish 2023-12-24T17:25:31.514 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:31.622 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2023-12-24T17:25:35.625 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T17:25:35.734 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-12-24T17:25:35.835 INFO:teuthology.orchestra.console.smithi133:Power on completed 2023-12-24T17:25:35.936 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish 2023-12-24T17:27:58.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:28:06.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:28:09.969 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-12-24T17:28:15.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:28:19.029 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-12-24T17:28:25.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:28:28.085 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-12-24T17:28:34.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:28:37.141 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-12-24T17:28:43.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:28:46.197 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-12-24T17:28:52.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:28:55.253 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-12-24T17:28:58.715 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9680d2aba1e76f64ade295c81599c97907457848/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_9680d2aba1e76f64ade295c81599c97907457848/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 427, in connect server_key = t.get_remote_server_key() File "/home/teuthworker/src/git.ceph.com_teuthology_9680d2aba1e76f64ade295c81599c97907457848/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 879, in get_remote_server_key raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2023-12-24T17:29:01.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:29:04.309 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-12-24T17:29:10.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:29:44.982 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-12-24T17:30:02.738 WARNING:teuthology.provision.fog.smithi181:timed out 2023-12-24T17:30:08.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:30:09.195 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-24T17:30:09.196 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-24T17:30:09.519 INFO:teuthology.provision.fog.smithi181:Node is ready 2023-12-24T17:30:09.519 DEBUG:teuthology.orchestra.run.smithi181:> hostname 2023-12-24T17:30:09.534 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com 2023-12-24T17:30:09.534 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts 2023-12-24T17:30:09.589 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181 2023-12-24T17:30:09.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-12-24T17:30:09.727 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts 2023-12-24T17:30:09.756 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts 2023-12-24T17:30:09.825 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname 2023-12-24T17:30:09.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-12-24T17:30:09.960 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-12-24T17:30:10.016 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-12-24T17:30:10.016 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2023-12-24T17:30:10.016 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-12-24T17:30:10.016 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-12-24T17:30:10.016 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2023-12-24T17:30:10.016 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2023-12-24T17:30:10.017 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-24T17:30:10.017 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-12-24T17:30:10.017 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2023-12-24T17:30:10.017 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-24T17:30:10.017 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-12-24T17:30:10.018 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-24T17:30:10.019 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-24T17:30:10.019 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-24T17:30:10.020 INFO:teuthology.provision.fog.smithi181:Deploy complete! 2023-12-24T17:30:10.022 DEBUG:teuthology.parallel:result is None 2023-12-24T17:30:10.314 WARNING:teuthology.provision.fog.smithi133:timed out 2023-12-24T17:30:16.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:30:16.563 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-24T17:30:16.563 DEBUG:teuthology.orchestra.run.smithi133:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-24T17:30:16.876 INFO:teuthology.provision.fog.smithi133:Node is ready 2023-12-24T17:30:16.876 DEBUG:teuthology.orchestra.run.smithi133:> hostname 2023-12-24T17:30:16.891 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com 2023-12-24T17:30:16.891 DEBUG:teuthology.orchestra.run.smithi133:> grep smithi133.front.sepia.ceph.com /etc/hosts 2023-12-24T17:30:16.946 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133 2023-12-24T17:30:16.947 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname smithi133 2023-12-24T17:30:17.044 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/smithi133.front.sepia.ceph.com/smithi133/g' /etc/hosts 2023-12-24T17:30:17.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/172.21.15.133/172.21.15.133/g' /etc/hosts 2023-12-24T17:30:17.140 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/smithi133.front.sepia.ceph.com/smithi133/g' /etc/hostname 2023-12-24T17:30:17.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname smithi133 2023-12-24T17:30:17.273 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-12-24T17:30:17.328 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2023-12-24T17:30:17.328 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2023-12-24T17:30:17.328 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2023-12-24T17:30:17.328 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-24T17:30:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-24T17:30:17.330 INFO:teuthology.provision.fog.smithi133:Deploy complete! 2023-12-24T17:30:17.330 DEBUG:teuthology.parallel:result is None 2023-12-24T17:30:17.490 INFO:teuthology.lock.ops:Checking smithi133.front.sepia.ceph.com 2023-12-24T17:30:17.491 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-24T17:30:17.515 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com 2023-12-24T17:30:17.515 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-24T17:30:17.533 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: set os type and version on server 2023-12-24T17:30:17.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-12-24T17:30:17.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:30:17.616 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-12-24T17:30:17.687 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-12-24T17:30:17.688 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2023-12-24T17:30:17.742 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2023-12-24T17:30:17.743 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-24T17:30:17.797 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2023-12-24T17:30:17.798 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-24T17:30:17.798 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-24T17:30:17.798 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-24T17:30:17.798 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-12-24T17:30:17.824 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server 2023-12-24T17:30:17.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-12-24T17:30:17.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-24T17:30:17.908 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-12-24T17:30:17.978 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-12-24T17:30:17.978 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-12-24T17:30:18.034 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-12-24T17:30:18.034 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-12-24T17:30:18.089 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-24T17:30:18.090 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-24T17:30:18.091 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-12-24T17:30:18.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:30:18.219 INFO:teuthology.dispatcher.supervisor:Running job 7500235 2023-12-24T17:30:18.220 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9680d2aba1e76f64ade295c81599c97907457848/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-24_15:36:42-rados-wip-yuri10-testing-2023-12-23-0824-distro-default-smithi/7500235 --name yuriw-2023-12-24_15:36:42-rados-wip-yuri10-testing-2023-12-23-0824-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} -- /home/teuthworker/archive/yuriw-2023-12-24_15:36:42-rados-wip-yuri10-testing-2023-12-23-0824-distro-default-smithi/7500235/orig.config.yaml 2023-12-24T17:30:18.223 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-12-24_15:36:42-rados-wip-yuri10-testing-2023-12-23-0824-distro-default-smithi/7500235 2023-12-24T17:30:18.223 INFO:teuthology.dispatcher.supervisor:Job PID: 3870119 2023-12-24T17:30:18.223 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-24T17:32:18.322 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:34:18.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:36:18.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:38:18.818 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:40:18.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:42:19.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:44:19.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:46:19.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:48:19.638 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:50:19.946 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:52:20.118 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:54:20.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:56:20.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T17:58:20.626 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:00:20.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:02:21.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:04:21.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:06:21.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:08:21.560 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:10:21.734 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:12:21.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:14:22.066 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:16:22.222 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:18:22.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:20:22.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:22:22.732 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:24:22.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:26:23.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:28:23.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:30:23.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:32:23.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:34:23.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:36:23.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:38:24.150 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:40:24.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:42:24.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:44:24.641 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:46:24.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:48:24.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:50:25.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:52:25.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:54:25.482 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:56:25.657 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T18:58:25.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:00:26.008 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:02:26.174 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:04:26.338 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:06:26.506 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:08:26.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:10:26.841 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:12:27.003 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:14:27.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:16:27.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-24T19:16:27.416 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-24T19:16:27.600 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-24T19:16:27.600 INFO:teuthology.nuke:Checking targets against current locks 2023-12-24T19:16:27.648 DEBUG:teuthology.nuke:shortname: smithi133 2023-12-24T19:16:27.648 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-24T19:16:27.675 DEBUG:teuthology.nuke:shortname: smithi181 2023-12-24T19:16:27.675 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-24T19:16:27.679 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-24_15:36:42-rados-wip-yuri10-testing-2023-12-23-0824-distro-default-smithi/7500235', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-24 17:25:13.378142', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUmZN0dl9eIH56W0u6yP6OaHKSvN9qbvx+Cgp2pdO1myCODtO910R3E1lawVJcFeei/s5miIhGrk/iKbcp6xE0='} 2023-12-24T19:16:27.699 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-24_15:36:42-rados-wip-yuri10-testing-2023-12-23-0824-distro-default-smithi/7500235', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-24 17:25:13.380103', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCeCK8N1ifJhN4wJdghQ768ueit9t5qxYpV8tXU9yRWhk7GC8cYfYIK+NJQubAU2JRKk3Xx4aqgeTuZIfWgvWcQ='} 2023-12-24T19:16:27.731 INFO:teuthology.orchestra.console.smithi133:Power off 2023-12-24T19:16:27.731 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-24T19:16:27.753 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2023-12-24T19:16:27.754 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T19:16:27.772 INFO:teuthology.orchestra.console.smithi181:Power off 2023-12-24T19:16:27.772 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-24T19:16:27.780 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2023-12-24T19:16:27.793 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-12-24T19:16:27.793 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-24T19:16:27.815 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-12-24T19:16:27.881 INFO:teuthology.orchestra.console.smithi133:Power off completed 2023-12-24T19:16:27.917 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-12-24T19:16:28.030 INFO:teuthology.lock.ops:unlocked: smithi133.front.sepia.ceph.com 2023-12-24T19:16:28.063 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com