2023-11-10T03:28:02.558 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-10T03:28:02.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:28:02.695 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3282382670>, signals=[15]) 2023-11-10T03:28:02.696 INFO:teuthology.lock.ops:Start node 'smithi044.front.sepia.ceph.com' reimaging 2023-11-10T03:28:02.697 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: reset os type and version on server 2023-11-10T03:28:02.697 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2023-11-10T03:28:02.719 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging 2023-11-10T03:28:02.720 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server 2023-11-10T03:28:02.720 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2023-11-10T03:28:02.721 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /host {"name": "smithi044"} 2023-11-10T03:28:02.745 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /host {"name": "smithi164"} 2023-11-10T03:28:02.796 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-10T03:28:02.818 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-10T03:28:02.837 DEBUG:teuthology.provision.fog.smithi044:Requesting image smithi_centos_8.stream (ID 3) 2023-11-10T03:28:02.837 DEBUG:teuthology.provision.fog.smithi044:FOG request: PUT /host/149 {"imageID": 3} 2023-11-10T03:28:02.857 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_centos_8.stream (ID 3) 2023-11-10T03:28:02.858 DEBUG:teuthology.provision.fog.smithi164:FOG request: PUT /host/269 {"imageID": 3} 2023-11-10T03:28:02.878 INFO:teuthology.provision.fog.smithi044:Scheduling deploy of centos 8.stream 2023-11-10T03:28:02.878 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:28:02.899 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 8.stream 2023-11-10T03:28:02.899 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:28:02.935 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /tasktype {"name": "deploy"} 2023-11-10T03:28:02.940 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /tasktype {"name": "deploy"} 2023-11-10T03:28:02.970 DEBUG:teuthology.provision.fog.smithi164:FOG request: POST /host/269/task {"taskTypeID": 1} 2023-11-10T03:28:02.973 DEBUG:teuthology.provision.fog.smithi044:FOG request: POST /host/149/task {"taskTypeID": 1} 2023-11-10T03:28:03.024 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:28:03.028 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:28:03.110 INFO:teuthology.orchestra.console.smithi164:Power off 2023-11-10T03:28:03.111 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T03:28:03.124 INFO:teuthology.orchestra.console.smithi044:Power off 2023-11-10T03:28:03.125 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T03:28:03.134 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2023-11-10T03:28:03.135 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:03.146 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2023-11-10T03:28:03.147 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:03.158 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2023-11-10T03:28:03.168 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-11-10T03:28:07.162 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:07.171 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:07.273 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2023-11-10T03:28:07.281 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-11-10T03:28:11.278 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:11.285 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:11.388 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2023-11-10T03:28:11.395 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-11-10T03:28:11.490 INFO:teuthology.orchestra.console.smithi164:Power off completed 2023-11-10T03:28:11.497 INFO:teuthology.orchestra.console.smithi044:Power off completed 2023-11-10T03:28:11.591 INFO:teuthology.orchestra.console.smithi164:Power on 2023-11-10T03:28:11.592 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-10T03:28:11.599 INFO:teuthology.orchestra.console.smithi044:Power on 2023-11-10T03:28:11.599 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-10T03:28:11.612 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On 2023-11-10T03:28:11.613 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:11.620 DEBUG:teuthology.orchestra.console.smithi044:power on output: Chassis Power Control: Up/On 2023-11-10T03:28:11.621 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:11.633 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2023-11-10T03:28:11.642 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-11-10T03:28:15.636 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:15.644 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:15.746 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2023-11-10T03:28:15.752 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-11-10T03:28:19.751 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:19.758 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:28:19.861 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2023-11-10T03:28:19.868 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-11-10T03:28:19.962 INFO:teuthology.orchestra.console.smithi164:Power on completed 2023-11-10T03:28:19.969 INFO:teuthology.orchestra.console.smithi044:Power on completed 2023-11-10T03:28:20.064 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2023-11-10T03:28:20.064 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:28:20.071 INFO:teuthology.provision.fog.smithi044:Waiting for deploy to finish 2023-11-10T03:28:20.071 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:28:35.139 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:28:35.140 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:28:50.214 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:28:50.216 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:29:05.282 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:29:05.283 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:29:20.350 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:29:20.352 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:29:35.422 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:29:35.424 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:29:50.518 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:29:50.520 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:30:05.591 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:30:05.593 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:30:20.674 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:30:20.676 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:30:35.773 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:30:35.775 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:30:50.853 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:30:50.854 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-11-10T03:30:50.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:05.946 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-11-10T03:31:06.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:09.306 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-11-10T03:31:15.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:18.394 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-11-10T03:31:24.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:24.411 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-11-10T03:31:27.454 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-11-10T03:31:30.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:33.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:33.466 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-11-10T03:31:36.538 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-11-10T03:31:39.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:42.522 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-11-10T03:31:42.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:48.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:31:51.610 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-11-10T03:31:57.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:32:00.666 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-11-10T03:32:06.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:32:42.578 WARNING:teuthology.provision.fog.smithi044:timed out 2023-11-10T03:32:48.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:32:48.841 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-10T03:32:48.842 DEBUG:teuthology.orchestra.run.smithi044:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-10T03:32:49.067 INFO:teuthology.provision.fog.smithi044:Node is ready 2023-11-10T03:32:49.067 DEBUG:teuthology.orchestra.run.smithi044:> hostname 2023-11-10T03:32:49.131 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044.front.sepia.ceph.com 2023-11-10T03:32:49.132 DEBUG:teuthology.orchestra.run.smithi044:> grep smithi044.front.sepia.ceph.com /etc/hosts 2023-11-10T03:32:49.195 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44 smithi044.front.sepia.ceph.com smithi044 2023-11-10T03:32:49.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2023-11-10T03:32:49.300 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hosts 2023-11-10T03:32:49.332 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/172.21.15.44/172.21.15.44/g' /etc/hosts 2023-11-10T03:32:49.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hostname 2023-11-10T03:32:49.478 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2023-11-10T03:32:49.550 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2023-11-10T03:32:49.612 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2023-11-10T03:32:49.613 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8" 2023-11-10T03:32:49.613 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2023-11-10T03:32:49.613 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2023-11-10T03:32:49.614 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8" 2023-11-10T03:32:49.614 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2023-11-10T03:32:49.614 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-10T03:32:49.615 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2023-11-10T03:32:49.615 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-10T03:32:49.615 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2023-11-10T03:32:49.616 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T03:32:49.616 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-10T03:32:49.617 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T03:32:49.617 INFO:teuthology.provision.fog.smithi044:Deploy complete! 2023-11-10T03:32:49.619 DEBUG:teuthology.parallel:result is None 2023-11-10T03:33:06.686 WARNING:teuthology.provision.fog.smithi164:timed out 2023-11-10T03:33:12.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:33:12.904 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-10T03:33:12.904 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-10T03:33:13.144 INFO:teuthology.provision.fog.smithi164:Node is ready 2023-11-10T03:33:13.144 DEBUG:teuthology.orchestra.run.smithi164:> hostname 2023-11-10T03:33:13.163 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2023-11-10T03:33:13.164 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164.front.sepia.ceph.com /etc/hosts 2023-11-10T03:33:13.223 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2023-11-10T03:33:13.223 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164 2023-11-10T03:33:13.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hosts 2023-11-10T03:33:13.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts 2023-11-10T03:33:13.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hostname 2023-11-10T03:33:13.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164 2023-11-10T03:33:13.565 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2023-11-10T03:33:13.626 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2023-11-10T03:33:13.627 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8" 2023-11-10T03:33:13.628 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2023-11-10T03:33:13.628 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2023-11-10T03:33:13.628 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8" 2023-11-10T03:33:13.629 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2023-11-10T03:33:13.629 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-10T03:33:13.629 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2023-11-10T03:33:13.629 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-10T03:33:13.629 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2023-11-10T03:33:13.630 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T03:33:13.630 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-10T03:33:13.630 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T03:33:13.630 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2023-11-10T03:33:13.631 DEBUG:teuthology.parallel:result is None 2023-11-10T03:33:13.783 INFO:teuthology.lock.ops:Checking smithi044.front.sepia.ceph.com 2023-11-10T03:33:13.784 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-10T03:33:13.808 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2023-11-10T03:33:13.808 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-10T03:33:13.834 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: set os type and version on server 2023-11-10T03:33:13.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2023-11-10T03:33:13.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:33:13.905 DEBUG:teuthology.orchestra.run.smithi044:> true 2023-11-10T03:33:14.000 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2023-11-10T03:33:14.000 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2023-11-10T03:33:14.060 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2023-11-10T03:33:14.060 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2023-11-10T03:33:14.120 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2023-11-10T03:33:14.120 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8" 2023-11-10T03:33:14.120 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2023-11-10T03:33:14.121 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2023-11-10T03:33:14.121 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8" 2023-11-10T03:33:14.121 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2023-11-10T03:33:14.121 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-10T03:33:14.122 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2023-11-10T03:33:14.122 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-10T03:33:14.122 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2023-11-10T03:33:14.122 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T03:33:14.122 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-10T03:33:14.123 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T03:33:14.123 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2023-11-10T03:33:14.150 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2023-11-10T03:33:14.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-11-10T03:33:14.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T03:33:14.216 DEBUG:teuthology.orchestra.run.smithi164:> true 2023-11-10T03:33:14.284 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-11-10T03:33:14.285 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2023-11-10T03:33:14.342 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2023-11-10T03:33:14.343 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2023-11-10T03:33:14.400 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2023-11-10T03:33:14.400 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8" 2023-11-10T03:33:14.400 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2023-11-10T03:33:14.401 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2023-11-10T03:33:14.401 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8" 2023-11-10T03:33:14.401 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2023-11-10T03:33:14.401 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-10T03:33:14.401 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2023-11-10T03:33:14.402 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-10T03:33:14.402 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2023-11-10T03:33:14.402 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T03:33:14.402 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-10T03:33:14.403 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T03:33:14.403 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2023-11-10T03:33:14.425 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:33:14.517 INFO:teuthology.dispatcher.supervisor:Running job 7453506 2023-11-10T03:33:14.517 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-09_21:43:29-rados-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453506 --name yuriw-2023-11-09_21:43:29-rados-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi --description rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/failover} -- /home/teuthworker/archive/yuriw-2023-11-09_21:43:29-rados-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453506/orig.config.yaml 2023-11-10T03:33:14.520 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-09_21:43:29-rados-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453506 2023-11-10T03:33:14.523 INFO:teuthology.dispatcher.supervisor:Job PID: 2949522 2023-11-10T03:33:14.523 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-10T03:35:14.623 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:37:14.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:39:14.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:41:15.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:43:15.307 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:45:15.483 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:47:15.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:49:15.827 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:51:15.995 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T03:51:16.069 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-10T03:51:16.218 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-10T03:51:16.219 INFO:teuthology.nuke:Checking targets against current locks 2023-11-10T03:51:16.271 DEBUG:teuthology.nuke:shortname: smithi044 2023-11-10T03:51:16.272 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-10T03:51:16.292 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-09_21:43:29-rados-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-10 03:28:01.785611', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCfhXKOvx4PgwI6dBzQdBOjM0fUJaCakt7m7N8rqYT4gPGZtYSJpjIBHbprKpYlxTLA4xi/tHF9xc/OrLDPWtBk='} 2023-11-10T03:51:16.295 DEBUG:teuthology.nuke:shortname: smithi164 2023-11-10T03:51:16.296 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-10T03:51:16.320 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-09_21:43:29-rados-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-10 03:28:01.787077', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeEGkaTb7d8gSwN2BUxnMnO2fJsg4YDlaTJUynCCmZ+BGAFiwTggPkkhGrvE/Pw3eoXsZNKb93M8SgEzFKy9d4='} 2023-11-10T03:51:16.348 INFO:teuthology.orchestra.console.smithi044:Power off 2023-11-10T03:51:16.348 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T03:51:16.371 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2023-11-10T03:51:16.371 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:51:16.386 INFO:teuthology.orchestra.console.smithi164:Power off 2023-11-10T03:51:16.387 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T03:51:16.397 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-11-10T03:51:16.410 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2023-11-10T03:51:16.410 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T03:51:16.432 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2023-11-10T03:51:16.499 INFO:teuthology.orchestra.console.smithi044:Power off completed 2023-11-10T03:51:16.534 INFO:teuthology.orchestra.console.smithi164:Power off completed 2023-11-10T03:51:16.652 INFO:teuthology.lock.ops:unlocked: smithi044.front.sepia.ceph.com 2023-11-10T03:51:16.685 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com