2023-11-11T00:17:12.124 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-11T00:17:12.126 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:17:12.263 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fde5e1a0670>, signals=[15]) 2023-11-11T00:17:12.265 INFO:teuthology.lock.ops:Start node 'smithi100.front.sepia.ceph.com' reimaging 2023-11-11T00:17:12.266 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: reset os type and version on server 2023-11-11T00:17:12.267 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2023-11-11T00:17:12.292 INFO:teuthology.lock.ops:Start node 'smithi138.front.sepia.ceph.com' reimaging 2023-11-11T00:17:12.292 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: reset os type and version on server 2023-11-11T00:17:12.293 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2023-11-11T00:17:12.294 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /host {"name": "smithi100"} 2023-11-11T00:17:12.315 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /host {"name": "smithi138"} 2023-11-11T00:17:12.380 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-11T00:17:12.392 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-11T00:17:12.425 DEBUG:teuthology.provision.fog.smithi100:Requesting image smithi_centos_9.stream (ID 10) 2023-11-11T00:17:12.425 DEBUG:teuthology.provision.fog.smithi100:FOG request: PUT /host/205 {"imageID": 10} 2023-11-11T00:17:12.434 DEBUG:teuthology.provision.fog.smithi138:Requesting image smithi_centos_9.stream (ID 10) 2023-11-11T00:17:12.435 DEBUG:teuthology.provision.fog.smithi138:FOG request: PUT /host/243 {"imageID": 10} 2023-11-11T00:17:12.474 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of centos 9.stream 2023-11-11T00:17:12.475 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:17:12.484 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of centos 9.stream 2023-11-11T00:17:12.484 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:17:12.543 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /tasktype {"name": "deploy"} 2023-11-11T00:17:12.552 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /tasktype {"name": "deploy"} 2023-11-11T00:17:12.577 DEBUG:teuthology.provision.fog.smithi100:FOG request: POST /host/205/task {"taskTypeID": 1} 2023-11-11T00:17:12.591 DEBUG:teuthology.provision.fog.smithi138:FOG request: POST /host/243/task {"taskTypeID": 1} 2023-11-11T00:17:12.632 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:17:12.637 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:17:12.742 INFO:teuthology.orchestra.console.smithi100:Power off 2023-11-11T00:17:12.743 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-11T00:17:12.761 INFO:teuthology.orchestra.console.smithi138:Power off 2023-11-11T00:17:12.763 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-11T00:17:12.774 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2023-11-11T00:17:12.774 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:12.791 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2023-11-11T00:17:12.791 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:12.803 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-11-11T00:17:12.820 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2023-11-11T00:17:16.807 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:16.821 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:16.917 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-11-11T00:17:16.930 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2023-11-11T00:17:20.921 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:20.932 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:21.030 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-11-11T00:17:21.041 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2023-11-11T00:17:21.132 INFO:teuthology.orchestra.console.smithi100:Power off completed 2023-11-11T00:17:21.233 INFO:teuthology.orchestra.console.smithi100:Power on 2023-11-11T00:17:21.233 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-11T00:17:21.255 DEBUG:teuthology.orchestra.console.smithi100:power on output: Chassis Power Control: Up/On 2023-11-11T00:17:21.256 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:21.276 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-11-11T00:17:25.046 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:25.154 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2023-11-11T00:17:25.256 INFO:teuthology.orchestra.console.smithi138:Power off completed 2023-11-11T00:17:25.277 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:25.357 INFO:teuthology.orchestra.console.smithi138:Power on 2023-11-11T00:17:25.358 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-11T00:17:25.384 DEBUG:teuthology.orchestra.console.smithi138:power on output: Chassis Power Control: Up/On 2023-11-11T00:17:25.384 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:25.393 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-11-11T00:17:25.405 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2023-11-11T00:17:29.398 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:29.407 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T00:17:29.508 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-11-11T00:17:29.515 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2023-11-11T00:17:29.610 INFO:teuthology.orchestra.console.smithi100:Power on completed 2023-11-11T00:17:29.617 INFO:teuthology.orchestra.console.smithi138:Power on completed 2023-11-11T00:17:29.711 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish 2023-11-11T00:17:29.712 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:17:29.718 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2023-11-11T00:17:29.718 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:17:44.802 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:17:44.804 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:17:59.905 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:17:59.907 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:18:15.010 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:18:15.012 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:18:30.098 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:18:30.100 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:18:45.186 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:18:45.188 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:19:00.266 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:19:00.268 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:19:15.346 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:19:15.348 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:19:30.441 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:19:30.443 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:19:45.526 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:19:45.529 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:20:00.622 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-11-11T00:20:00.624 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-11-11T00:20:00.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:00.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:03.774 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-11T00:20:03.775 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-11-11T00:20:09.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:09.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:12.858 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-11T00:20:12.859 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-11-11T00:20:18.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:18.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:21.914 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-11T00:20:21.946 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-11-11T00:20:27.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:27.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:30.970 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-11T00:20:31.006 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-11-11T00:20:36.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:37.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:40.026 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-11T00:20:40.058 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-11-11T00:20:46.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:46.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:49.118 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-11T00:20:49.119 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-11-11T00:20:55.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:55.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:20:58.206 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-11T00:20:58.207 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-11-11T00:21:04.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:21:04.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:22:04.275 WARNING:teuthology.provision.fog.smithi100:timed out 2023-11-11T00:22:04.277 WARNING:teuthology.provision.fog.smithi138:timed out 2023-11-11T00:22:10.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:22:10.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:22:10.521 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-11T00:22:10.522 DEBUG:teuthology.orchestra.run.smithi138:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-11T00:22:10.558 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-11T00:22:10.558 DEBUG:teuthology.orchestra.run.smithi100:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-11T00:22:10.849 INFO:teuthology.provision.fog.smithi138:Node is ready 2023-11-11T00:22:10.849 DEBUG:teuthology.orchestra.run.smithi138:> hostname 2023-11-11T00:22:10.864 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2023-11-11T00:22:10.865 DEBUG:teuthology.orchestra.run.smithi138:> grep smithi138.front.sepia.ceph.com /etc/hosts 2023-11-11T00:22:10.904 INFO:teuthology.provision.fog.smithi100:Node is ready 2023-11-11T00:22:10.904 DEBUG:teuthology.orchestra.run.smithi100:> hostname 2023-11-11T00:22:10.919 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com 2023-11-11T00:22:10.919 DEBUG:teuthology.orchestra.run.smithi100:> grep smithi100.front.sepia.ceph.com /etc/hosts 2023-11-11T00:22:10.921 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2023-11-11T00:22:10.921 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname smithi138 2023-11-11T00:22:10.975 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100 2023-11-11T00:22:10.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2023-11-11T00:22:11.024 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/smithi138.front.sepia.ceph.com/smithi138/g' /etc/hosts 2023-11-11T00:22:11.055 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/172.21.15.138/172.21.15.138/g' /etc/hosts 2023-11-11T00:22:11.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi100.front.sepia.ceph.com/smithi100/g' /etc/hosts 2023-11-11T00:22:11.104 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/172.21.15.100/172.21.15.100/g' /etc/hosts 2023-11-11T00:22:11.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/smithi138.front.sepia.ceph.com/smithi138/g' /etc/hostname 2023-11-11T00:22:11.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi100.front.sepia.ceph.com/smithi100/g' /etc/hostname 2023-11-11T00:22:11.188 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname smithi138 2023-11-11T00:22:11.238 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2023-11-11T00:22:11.254 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2023-11-11T00:22:11.305 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2023-11-11T00:22:11.307 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2023-11-11T00:22:11.307 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2023-11-11T00:22:11.307 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2023-11-11T00:22:11.307 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2023-11-11T00:22:11.308 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2023-11-11T00:22:11.308 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2023-11-11T00:22:11.308 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-11T00:22:11.308 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2023-11-11T00:22:11.308 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2023-11-11T00:22:11.309 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-11T00:22:11.309 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2023-11-11T00:22:11.309 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-11T00:22:11.309 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-11T00:22:11.309 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-11T00:22:11.310 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2023-11-11T00:22:11.312 DEBUG:teuthology.parallel:result is None 2023-11-11T00:22:11.359 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2023-11-11T00:22:11.359 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2023-11-11T00:22:11.359 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2023-11-11T00:22:11.359 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2023-11-11T00:22:11.360 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2023-11-11T00:22:11.360 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2023-11-11T00:22:11.360 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-11T00:22:11.360 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2023-11-11T00:22:11.360 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2023-11-11T00:22:11.361 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-11T00:22:11.361 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2023-11-11T00:22:11.361 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-11T00:22:11.361 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-11T00:22:11.361 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-11T00:22:11.362 INFO:teuthology.provision.fog.smithi100:Deploy complete! 2023-11-11T00:22:11.363 DEBUG:teuthology.parallel:result is None 2023-11-11T00:22:11.527 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com 2023-11-11T00:22:11.527 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-11T00:22:11.548 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2023-11-11T00:22:11.548 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-11T00:22:11.574 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server 2023-11-11T00:22:11.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-11-11T00:22:11.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:22:11.658 DEBUG:teuthology.orchestra.run.smithi100:> true 2023-11-11T00:22:11.724 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-11-11T00:22:11.725 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2023-11-11T00:22:11.779 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2023-11-11T00:22:11.780 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2023-11-11T00:22:11.835 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2023-11-11T00:22:11.835 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2023-11-11T00:22:11.835 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2023-11-11T00:22:11.836 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2023-11-11T00:22:11.836 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2023-11-11T00:22:11.836 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2023-11-11T00:22:11.836 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-11T00:22:11.836 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2023-11-11T00:22:11.837 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2023-11-11T00:22:11.837 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-11T00:22:11.837 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2023-11-11T00:22:11.837 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-11T00:22:11.837 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-11T00:22:11.838 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-11T00:22:11.838 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2023-11-11T00:22:11.856 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2023-11-11T00:22:11.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-11-11T00:22:11.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-11T00:22:11.977 DEBUG:teuthology.orchestra.run.smithi138:> true 2023-11-11T00:22:12.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-11-11T00:22:12.049 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2023-11-11T00:22:12.102 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2023-11-11T00:22:12.102 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2023-11-11T00:22:12.153 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2023-11-11T00:22:12.154 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2023-11-11T00:22:12.154 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2023-11-11T00:22:12.154 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2023-11-11T00:22:12.154 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2023-11-11T00:22:12.155 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2023-11-11T00:22:12.155 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-11T00:22:12.155 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2023-11-11T00:22:12.155 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2023-11-11T00:22:12.155 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-11T00:22:12.156 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2023-11-11T00:22:12.156 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-11T00:22:12.156 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-11T00:22:12.156 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-11T00:22:12.157 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2023-11-11T00:22:12.181 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:22:12.283 INFO:teuthology.dispatcher.supervisor:Running job 7454991 2023-11-11T00:22:12.283 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-10_19:52:04-fs-wip-yuri5-testing-2023-11-10-0828-distro-default-smithi/7454991 --name yuriw-2023-11-10_19:52:04-fs-wip-yuri5-testing-2023-11-10-0828-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} -- /home/teuthworker/archive/yuriw-2023-11-10_19:52:04-fs-wip-yuri5-testing-2023-11-10-0828-distro-default-smithi/7454991/orig.config.yaml 2023-11-11T00:22:12.286 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-10_19:52:04-fs-wip-yuri5-testing-2023-11-10-0828-distro-default-smithi/7454991 2023-11-11T00:22:12.287 INFO:teuthology.dispatcher.supervisor:Job PID: 3908956 2023-11-11T00:22:12.287 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-11T00:24:12.387 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:26:12.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:28:12.715 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:30:12.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:32:13.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:34:13.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:36:13.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:38:13.538 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:40:13.704 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:42:13.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:44:14.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:46:14.208 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:48:14.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:50:14.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:52:14.695 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:54:14.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:56:15.099 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T00:58:15.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T01:00:15.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T01:02:15.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T01:04:15.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T01:06:15.952 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T01:08:16.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T01:10:16.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T01:12:16.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-11T01:12:16.508 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-11T01:12:16.729 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-11T01:12:16.730 INFO:teuthology.nuke:Checking targets against current locks 2023-11-11T01:12:16.782 DEBUG:teuthology.nuke:shortname: smithi100 2023-11-11T01:12:16.783 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-11T01:12:16.805 DEBUG:teuthology.nuke:shortname: smithi138 2023-11-11T01:12:16.806 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-11T01:12:16.809 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-10_19:52:04-fs-wip-yuri5-testing-2023-11-10-0828-distro-default-smithi/7454991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-11 00:17:11.236520', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFBbDvj9e4efA9s+WiO+jLc0kq0+wyZ79VAOOiwhG6wsSo7NlR603LBeKwclV/B9v+dRxmGfv5Wv/3rdU2hgW7Q='} 2023-11-11T01:12:16.830 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-10_19:52:04-fs-wip-yuri5-testing-2023-11-10-0828-distro-default-smithi/7454991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-11 00:17:11.238494', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0D9zfZws7kjEL1vOJQKuyj/TRcqGHxC5og6n2rZ9LKu9tdejEnQdGA+KSE+4Gb3v0VpaLIZKbzvI9P1pFH478='} 2023-11-11T01:12:16.861 INFO:teuthology.orchestra.console.smithi100:Power off 2023-11-11T01:12:16.861 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-11T01:12:16.885 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2023-11-11T01:12:16.885 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T01:12:16.900 INFO:teuthology.orchestra.console.smithi138:Power off 2023-11-11T01:12:16.900 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-11T01:12:16.910 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-11-11T01:12:16.924 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2023-11-11T01:12:16.924 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-11T01:12:16.946 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2023-11-11T01:12:17.011 INFO:teuthology.orchestra.console.smithi100:Power off completed 2023-11-11T01:12:17.048 INFO:teuthology.orchestra.console.smithi138:Power off completed 2023-11-11T01:12:17.166 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com 2023-11-11T01:12:17.198 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com