2023-10-21T03:01:31.389 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-21T03:01:31.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:01:31.523 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f77b26b5670>, signals=[15]) 2023-10-21T03:01:31.524 INFO:teuthology.lock.ops:Start node 'smithi046.front.sepia.ceph.com' reimaging 2023-10-21T03:01:31.524 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: reset os type and version on server 2023-10-21T03:01:31.525 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2023-10-21T03:01:31.547 INFO:teuthology.lock.ops:Start node 'smithi158.front.sepia.ceph.com' reimaging 2023-10-21T03:01:31.547 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: reset os type and version on server 2023-10-21T03:01:31.547 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2023-10-21T03:01:31.549 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /host {"name": "smithi046"} 2023-10-21T03:01:31.572 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /host {"name": "smithi158"} 2023-10-21T03:01:31.620 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-10-21T03:01:31.640 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-10-21T03:01:31.660 DEBUG:teuthology.provision.fog.smithi046:Requesting image smithi_centos_9.stream (ID 10) 2023-10-21T03:01:31.660 DEBUG:teuthology.provision.fog.smithi046:FOG request: PUT /host/151 {"imageID": 10} 2023-10-21T03:01:31.690 DEBUG:teuthology.provision.fog.smithi158:Requesting image smithi_centos_9.stream (ID 10) 2023-10-21T03:01:31.691 DEBUG:teuthology.provision.fog.smithi158:FOG request: PUT /host/263 {"imageID": 10} 2023-10-21T03:01:31.712 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of centos 9.stream 2023-10-21T03:01:31.713 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:01:31.733 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of centos 9.stream 2023-10-21T03:01:31.734 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:01:31.776 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /tasktype {"name": "deploy"} 2023-10-21T03:01:31.784 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /tasktype {"name": "deploy"} 2023-10-21T03:01:31.808 DEBUG:teuthology.provision.fog.smithi046:FOG request: POST /host/151/task {"taskTypeID": 1} 2023-10-21T03:01:31.820 DEBUG:teuthology.provision.fog.smithi158:FOG request: POST /host/263/task {"taskTypeID": 1} 2023-10-21T03:01:31.859 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:01:31.872 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:01:31.952 INFO:teuthology.orchestra.console.smithi046:Power off 2023-10-21T03:01:31.953 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T03:01:31.970 INFO:teuthology.orchestra.console.smithi158:Power off 2023-10-21T03:01:31.971 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T03:01:31.982 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2023-10-21T03:01:31.983 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:31.994 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off 2023-10-21T03:01:31.994 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:32.010 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-10-21T03:01:32.022 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-10-21T03:01:36.014 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:36.023 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:36.123 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-10-21T03:01:36.134 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-10-21T03:01:40.126 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:40.134 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:40.236 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2023-10-21T03:01:40.243 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-10-21T03:01:40.338 INFO:teuthology.orchestra.console.smithi046:Power off completed 2023-10-21T03:01:40.438 INFO:teuthology.orchestra.console.smithi046:Power on 2023-10-21T03:01:40.439 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-21T03:01:40.460 DEBUG:teuthology.orchestra.console.smithi046:power on output: Chassis Power Control: Up/On 2023-10-21T03:01:40.460 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:40.482 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2023-10-21T03:01:44.247 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:44.355 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-10-21T03:01:44.456 INFO:teuthology.orchestra.console.smithi158:Power off completed 2023-10-21T03:01:44.483 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:44.558 INFO:teuthology.orchestra.console.smithi158:Power on 2023-10-21T03:01:44.558 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-21T03:01:44.581 DEBUG:teuthology.orchestra.console.smithi158:power on output: Chassis Power Control: Up/On 2023-10-21T03:01:44.581 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:44.592 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2023-10-21T03:01:44.603 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-10-21T03:01:48.594 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:48.604 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:48.703 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-10-21T03:01:48.713 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-10-21T03:01:48.805 INFO:teuthology.orchestra.console.smithi046:Power on completed 2023-10-21T03:01:48.906 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish 2023-10-21T03:01:48.907 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:01:52.714 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T03:01:52.822 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-10-21T03:01:52.923 INFO:teuthology.orchestra.console.smithi158:Power on completed 2023-10-21T03:01:53.024 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish 2023-10-21T03:01:53.025 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:02:03.974 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:02:08.098 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:02:19.047 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:02:23.162 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:02:34.122 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:02:38.238 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:02:49.198 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:02:53.326 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:03:04.275 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:03:08.390 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:03:19.350 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:03:23.466 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:03:34.432 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:03:38.538 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:03:49.534 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:03:53.638 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:04:04.622 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:04:08.710 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:04:19.734 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-10-21T03:04:19.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:04:23.794 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:04:38.266 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-10-21T03:04:38.872 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:04:44.270 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:04:47.322 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-10-21T03:04:53.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:04:53.937 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-10-21T03:04:54.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:04:56.378 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-10-21T03:05:02.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:05:05.466 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-10-21T03:05:11.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:05:12.574 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-21T03:05:14.526 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-10-21T03:05:18.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:05:20.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:05:21.658 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-21T03:05:27.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:05:30.718 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-21T03:05:36.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:05:39.802 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-21T03:05:45.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:05:48.858 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-21T03:05:54.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:05:57.914 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-21T03:06:03.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:06:06.974 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-21T03:06:12.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:06:20.538 WARNING:teuthology.provision.fog.smithi046:timed out 2023-10-21T03:06:26.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:06:26.817 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-21T03:06:26.818 DEBUG:teuthology.orchestra.run.smithi046:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-21T03:06:27.177 INFO:teuthology.provision.fog.smithi046:Node is ready 2023-10-21T03:06:27.178 DEBUG:teuthology.orchestra.run.smithi046:> hostname 2023-10-21T03:06:27.193 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com 2023-10-21T03:06:27.193 DEBUG:teuthology.orchestra.run.smithi046:> grep smithi046.front.sepia.ceph.com /etc/hosts 2023-10-21T03:06:27.249 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2023-10-21T03:06:27.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname smithi046 2023-10-21T03:06:27.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/smithi046.front.sepia.ceph.com/smithi046/g' /etc/hosts 2023-10-21T03:06:27.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/172.21.15.46/172.21.15.46/g' /etc/hosts 2023-10-21T03:06:27.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/smithi046.front.sepia.ceph.com/smithi046/g' /etc/hostname 2023-10-21T03:06:27.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname smithi046 2023-10-21T03:06:27.625 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2023-10-21T03:06:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2023-10-21T03:06:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2023-10-21T03:06:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2023-10-21T03:06:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2023-10-21T03:06:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2023-10-21T03:06:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2023-10-21T03:06:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T03:06:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2023-10-21T03:06:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2023-10-21T03:06:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T03:06:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2023-10-21T03:06:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T03:06:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T03:06:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T03:06:27.683 INFO:teuthology.provision.fog.smithi046:Deploy complete! 2023-10-21T03:06:27.685 DEBUG:teuthology.parallel:result is None 2023-10-21T03:07:13.022 WARNING:teuthology.provision.fog.smithi158:timed out 2023-10-21T03:07:19.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:07:19.278 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-21T03:07:19.279 DEBUG:teuthology.orchestra.run.smithi158:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-21T03:07:19.609 INFO:teuthology.provision.fog.smithi158:Node is ready 2023-10-21T03:07:19.609 DEBUG:teuthology.orchestra.run.smithi158:> hostname 2023-10-21T03:07:19.624 INFO:teuthology.orchestra.run.smithi158.stdout:smithi158.front.sepia.ceph.com 2023-10-21T03:07:19.624 DEBUG:teuthology.orchestra.run.smithi158:> grep smithi158.front.sepia.ceph.com /etc/hosts 2023-10-21T03:07:19.680 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.158 smithi158.front.sepia.ceph.com smithi158 2023-10-21T03:07:19.680 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158 2023-10-21T03:07:19.781 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hosts 2023-10-21T03:07:19.810 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/172.21.15.158/172.21.15.158/g' /etc/hosts 2023-10-21T03:07:19.877 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hostname 2023-10-21T03:07:19.945 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158 2023-10-21T03:07:20.011 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2023-10-21T03:07:20.065 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream" 2023-10-21T03:07:20.065 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9" 2023-10-21T03:07:20.065 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos" 2023-10-21T03:07:20.066 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora" 2023-10-21T03:07:20.066 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9" 2023-10-21T03:07:20.066 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9" 2023-10-21T03:07:20.066 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T03:07:20.066 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2023-10-21T03:07:20.067 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon" 2023-10-21T03:07:20.067 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T03:07:20.067 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/" 2023-10-21T03:07:20.067 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T03:07:20.068 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T03:07:20.068 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T03:07:20.068 INFO:teuthology.provision.fog.smithi158:Deploy complete! 2023-10-21T03:07:20.069 DEBUG:teuthology.parallel:result is None 2023-10-21T03:07:20.241 INFO:teuthology.lock.ops:Checking smithi046.front.sepia.ceph.com 2023-10-21T03:07:20.242 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-21T03:07:20.279 INFO:teuthology.lock.ops:Checking smithi158.front.sepia.ceph.com 2023-10-21T03:07:20.279 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-21T03:07:20.322 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: set os type and version on server 2023-10-21T03:07:20.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-21T03:07:20.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:07:20.424 DEBUG:teuthology.orchestra.run.smithi046:> true 2023-10-21T03:07:20.621 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-21T03:07:20.621 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2023-10-21T03:07:20.676 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2023-10-21T03:07:20.677 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2023-10-21T03:07:20.731 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2023-10-21T03:07:20.731 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2023-10-21T03:07:20.732 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2023-10-21T03:07:20.732 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2023-10-21T03:07:20.732 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2023-10-21T03:07:20.733 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2023-10-21T03:07:20.733 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T03:07:20.733 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2023-10-21T03:07:20.733 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2023-10-21T03:07:20.733 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T03:07:20.733 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2023-10-21T03:07:20.734 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T03:07:20.734 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T03:07:20.734 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T03:07:20.735 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2023-10-21T03:07:20.762 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: set os type and version on server 2023-10-21T03:07:20.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-21T03:07:20.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T03:07:20.846 DEBUG:teuthology.orchestra.run.smithi158:> true 2023-10-21T03:07:20.913 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-21T03:07:20.913 DEBUG:teuthology.orchestra.run.smithi158:> uname -m 2023-10-21T03:07:20.968 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64 2023-10-21T03:07:20.969 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2023-10-21T03:07:21.023 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream" 2023-10-21T03:07:21.024 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9" 2023-10-21T03:07:21.024 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos" 2023-10-21T03:07:21.024 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora" 2023-10-21T03:07:21.024 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9" 2023-10-21T03:07:21.025 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9" 2023-10-21T03:07:21.025 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T03:07:21.025 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2023-10-21T03:07:21.025 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon" 2023-10-21T03:07:21.025 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T03:07:21.026 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/" 2023-10-21T03:07:21.026 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T03:07:21.026 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T03:07:21.026 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T03:07:21.027 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2023-10-21T03:07:21.046 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:07:21.142 INFO:teuthology.dispatcher.supervisor:Running job 7433294 2023-10-21T03:07:21.142 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-20_15:03:24-rados-wip-yuri3-testing-2023-10-18-1510-reef-distro-default-smithi/7433294 --name yuriw-2023-10-20_15:03:24-rados-wip-yuri3-testing-2023-10-18-1510-reef-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} -- /home/teuthworker/archive/yuriw-2023-10-20_15:03:24-rados-wip-yuri3-testing-2023-10-18-1510-reef-distro-default-smithi/7433294/orig.config.yaml 2023-10-21T03:07:21.145 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-10-20_15:03:24-rados-wip-yuri3-testing-2023-10-18-1510-reef-distro-default-smithi/7433294 2023-10-21T03:07:21.146 INFO:teuthology.dispatcher.supervisor:Job PID: 2112708 2023-10-21T03:07:21.147 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-21T03:09:21.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:11:21.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:13:21.579 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:15:21.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:17:21.939 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:19:22.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:21:22.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:23:22.452 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:25:22.635 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:27:22.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:29:22.900 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:31:23.075 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:33:23.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:35:23.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:37:23.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:39:23.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:41:23.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:43:24.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:45:24.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:47:24.417 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:49:24.579 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:51:24.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:53:24.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:55:25.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:57:25.307 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T03:59:25.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:01:25.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:03:25.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:05:25.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:07:26.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:09:26.312 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:11:26.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:13:26.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:15:26.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:17:26.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:19:27.134 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:21:27.331 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:23:27.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:25:27.591 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:27:27.764 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:29:28.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:31:28.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:33:28.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:35:28.628 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:37:28.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:39:28.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:41:29.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:43:29.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:45:29.474 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:47:29.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:49:29.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:51:29.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:53:30.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:55:30.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:57:30.490 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T04:59:30.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:01:30.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:03:30.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:05:31.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:07:31.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:09:31.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:11:31.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:13:31.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:15:32.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:17:32.215 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T05:17:32.283 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-21T05:17:32.441 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-21T05:17:32.442 INFO:teuthology.nuke:Checking targets against current locks 2023-10-21T05:17:32.493 DEBUG:teuthology.nuke:shortname: smithi046 2023-10-21T05:17:32.493 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T05:17:32.512 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-20_15:03:24-rados-wip-yuri3-testing-2023-10-18-1510-reef-distro-default-smithi/7433294', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 03:01:30.584071', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIr0/819subI5W355fZMjWJ24y7sq5waEAnDRtRegzWlcK8gv2N3g0mr9uw1xZ0Azd7ENU39o2VKYzrSujnprPw='} 2023-10-21T05:17:32.515 DEBUG:teuthology.nuke:shortname: smithi158 2023-10-21T05:17:32.516 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T05:17:32.533 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-20_15:03:24-rados-wip-yuri3-testing-2023-10-18-1510-reef-distro-default-smithi/7433294', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 03:01:30.582252', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNL3jDHMSUaf2me0QMC6xDTZDtfmeVRxyQB8ownBDVC41bT5++a9MTJxfFwVdjecgn/7eKNoCJ/jEdTSfTQmKM0='} 2023-10-21T05:17:32.567 INFO:teuthology.orchestra.console.smithi046:Power off 2023-10-21T05:17:32.568 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T05:17:32.593 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2023-10-21T05:17:32.593 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T05:17:32.614 INFO:teuthology.orchestra.console.smithi158:Power off 2023-10-21T05:17:32.615 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T05:17:32.628 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2023-10-21T05:17:32.642 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off 2023-10-21T05:17:32.643 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T05:17:32.665 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-10-21T05:17:32.729 INFO:teuthology.orchestra.console.smithi046:Power off completed 2023-10-21T05:17:32.767 INFO:teuthology.orchestra.console.smithi158:Power off completed 2023-10-21T05:17:32.882 INFO:teuthology.lock.ops:unlocked: smithi046.front.sepia.ceph.com 2023-10-21T05:17:32.916 INFO:teuthology.lock.ops:unlocked: smithi158.front.sepia.ceph.com