2023-12-09T23:56:45.302 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-09T23:56:45.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-09T23:56:45.464 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffab50bd550>, signals=[15]) 2023-12-09T23:56:45.465 INFO:teuthology.lock.ops:Start node 'smithi022.front.sepia.ceph.com' reimaging 2023-12-09T23:56:45.465 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: reset os type and version on server 2023-12-09T23:56:45.465 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2023-12-09T23:56:45.483 INFO:teuthology.lock.ops:Start node 'smithi040.front.sepia.ceph.com' reimaging 2023-12-09T23:56:45.483 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: reset os type and version on server 2023-12-09T23:56:45.483 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2023-12-09T23:56:45.499 INFO:teuthology.lock.ops:Start node 'smithi172.front.sepia.ceph.com' reimaging 2023-12-09T23:56:45.499 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: reset os type and version on server 2023-12-09T23:56:45.499 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-12-09T23:56:45.591 DEBUG:teuthology.provision.fog.smithi022:Requesting image smithi_centos_9.stream (ID 10) 2023-12-09T23:56:45.619 DEBUG:teuthology.provision.fog.smithi040:Requesting image smithi_centos_9.stream (ID 10) 2023-12-09T23:56:45.622 DEBUG:teuthology.provision.fog.smithi172:Requesting image smithi_centos_9.stream (ID 10) 2023-12-09T23:56:45.644 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of centos 9.stream 2023-12-09T23:56:45.662 INFO:teuthology.provision.fog.smithi040:Scheduling deploy of centos 9.stream 2023-12-09T23:56:45.666 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 9.stream 2023-12-09T23:56:45.921 INFO:teuthology.orchestra.console.smithi022:Power off 2023-12-09T23:56:45.922 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-09T23:56:45.934 INFO:teuthology.orchestra.console.smithi172:Power off 2023-12-09T23:56:45.934 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-09T23:56:45.943 INFO:teuthology.orchestra.console.smithi040:Power off 2023-12-09T23:56:45.944 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-09T23:56:45.953 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2023-12-09T23:56:45.953 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:45.959 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2023-12-09T23:56:45.959 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:45.967 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2023-12-09T23:56:45.968 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:45.976 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2023-12-09T23:56:45.982 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-12-09T23:56:45.987 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2023-12-09T23:56:46.083 INFO:teuthology.orchestra.console.smithi172:Power off completed 2023-12-09T23:56:46.088 INFO:teuthology.orchestra.console.smithi040:Power off completed 2023-12-09T23:56:46.184 INFO:teuthology.orchestra.console.smithi172:Power on 2023-12-09T23:56:46.184 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-09T23:56:46.192 INFO:teuthology.orchestra.console.smithi040:Power on 2023-12-09T23:56:46.192 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-09T23:56:46.206 DEBUG:teuthology.orchestra.console.smithi172:power on output: Chassis Power Control: Up/On 2023-12-09T23:56:46.206 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:46.218 DEBUG:teuthology.orchestra.console.smithi040:power on output: Chassis Power Control: Up/On 2023-12-09T23:56:46.219 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:46.227 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-12-09T23:56:46.247 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2023-12-09T23:56:49.978 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:50.086 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2023-12-09T23:56:50.229 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:50.248 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:50.338 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-12-09T23:56:50.356 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2023-12-09T23:56:54.091 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:54.198 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2023-12-09T23:56:54.299 INFO:teuthology.orchestra.console.smithi022:Power off completed 2023-12-09T23:56:54.339 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:54.357 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:54.400 INFO:teuthology.orchestra.console.smithi022:Power on 2023-12-09T23:56:54.400 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-09T23:56:54.422 DEBUG:teuthology.orchestra.console.smithi022:power on output: Chassis Power Control: Up/On 2023-12-09T23:56:54.422 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:54.442 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2023-12-09T23:56:54.448 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-12-09T23:56:54.499 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-12-09T23:56:54.548 INFO:teuthology.orchestra.console.smithi172:Power on completed 2023-12-09T23:56:54.600 INFO:teuthology.orchestra.console.smithi040:Power on completed 2023-12-09T23:56:54.649 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2023-12-09T23:56:54.701 INFO:teuthology.provision.fog.smithi040:Waiting for deploy to finish 2023-12-09T23:56:54.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:56:58.447 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:56:58.555 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2023-12-09T23:57:02.558 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-09T23:57:02.665 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2023-12-09T23:57:02.766 INFO:teuthology.orchestra.console.smithi022:Power on completed 2023-12-09T23:57:02.867 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish 2023-12-09T23:57:02.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:06.050 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-12-09T23:57:12.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:13.186 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-12-09T23:57:15.106 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-12-09T23:57:19.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:21.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:22.242 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-12-09T23:57:24.167 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-12-09T23:57:28.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:30.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:31.298 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-12-09T23:57:33.250 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-12-09T23:57:37.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:39.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:40.354 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-12-09T23:57:42.306 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-12-09T23:57:46.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:48.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:49.410 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-12-09T23:57:51.362 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-12-09T23:57:55.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:57:57.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:58:27.040 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:58:31.168 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:58:38.255 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:58:48.343 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:58:57.366 WARNING:teuthology.provision.fog.smithi022:timed out 2023-12-09T23:59:01.469 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:59:03.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-09T23:59:03.458 ERROR:teuthology.orchestra.connection:Error authenticating with smithi022.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:59:07.588 ERROR:teuthology.orchestra.connection:Error authenticating with smithi022.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:59:14.716 ERROR:teuthology.orchestra.connection:Error authenticating with smithi022.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:59:17.596 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:59:25.016 ERROR:teuthology.orchestra.connection:Error authenticating with smithi022.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-09T23:59:53.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:00:11.874 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-12-10T00:00:17.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:00:20.930 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-12-10T00:00:26.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:00:29.986 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-12-10T00:00:35.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:00:36.598 WARNING:teuthology.provision.fog.smithi040:timed out 2023-12-10T00:00:38.020 WARNING:teuthology.provision.fog.smithi022:timed out 2023-12-10T00:00:42.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:00:44.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:00:46.690 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-12-10T00:00:52.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:00:53.602 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-12-10T00:00:59.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:00:59.720 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-10T00:00:59.721 DEBUG:teuthology.orchestra.run.smithi022:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-10T00:01:01.607 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-10T00:01:01.607 DEBUG:teuthology.orchestra.run.smithi040:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-10T00:01:14.993 INFO:teuthology.provision.fog.smithi022:Node is ready 2023-12-10T00:01:14.993 DEBUG:teuthology.orchestra.run.smithi022:> hostname 2023-12-10T00:01:15.012 INFO:teuthology.orchestra.run.smithi022.stdout:smithi046 2023-12-10T00:01:15.012 DEBUG:teuthology.orchestra.run.smithi022:> grep smithi046 /etc/hosts 2023-12-10T00:01:15.069 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2023-12-10T00:01:15.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022 2023-12-10T00:01:15.209 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi046/smithi022/g' /etc/hosts 2023-12-10T00:01:15.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/172.21.15.46/172.21.15.22/g' /etc/hosts 2023-12-10T00:01:15.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi046/smithi022/g' /etc/hostname 2023-12-10T00:01:15.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022 2023-12-10T00:01:15.441 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2023-12-10T00:01:15.496 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2023-12-10T00:01:15.496 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2023-12-10T00:01:15.496 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2023-12-10T00:01:15.496 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2023-12-10T00:01:15.496 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2023-12-10T00:01:15.496 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2023-12-10T00:01:15.497 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T00:01:15.497 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2023-12-10T00:01:15.497 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2023-12-10T00:01:15.497 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T00:01:15.497 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2023-12-10T00:01:15.497 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T00:01:15.497 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T00:01:15.497 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T00:01:15.498 INFO:teuthology.provision.fog.smithi022:Deploy complete! 2023-12-10T00:01:15.499 DEBUG:teuthology.parallel:result is None 2023-12-10T00:01:16.930 INFO:teuthology.provision.fog.smithi040:Node is ready 2023-12-10T00:01:16.931 DEBUG:teuthology.orchestra.run.smithi040:> hostname 2023-12-10T00:01:16.949 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040.front.sepia.ceph.com 2023-12-10T00:01:16.950 DEBUG:teuthology.orchestra.run.smithi040:> grep smithi040.front.sepia.ceph.com /etc/hosts 2023-12-10T00:01:17.008 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40 smithi040.front.sepia.ceph.com smithi040 2023-12-10T00:01:17.008 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname smithi040 2023-12-10T00:01:17.142 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hosts 2023-12-10T00:01:17.175 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/172.21.15.40/172.21.15.40/g' /etc/hosts 2023-12-10T00:01:17.248 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hostname 2023-12-10T00:01:17.320 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname smithi040 2023-12-10T00:01:17.390 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2023-12-10T00:01:17.445 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2023-12-10T00:01:17.445 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2023-12-10T00:01:17.445 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2023-12-10T00:01:17.445 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2023-12-10T00:01:17.445 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T00:01:17.446 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T00:01:17.446 INFO:teuthology.provision.fog.smithi040:Deploy complete! 2023-12-10T00:01:17.447 DEBUG:teuthology.parallel:result is None 2023-12-10T00:01:35.990 WARNING:teuthology.provision.fog.smithi172:timed out 2023-12-10T00:01:41.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:01:42.227 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-10T00:01:42.227 DEBUG:teuthology.orchestra.run.smithi172:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-10T00:01:42.541 INFO:teuthology.provision.fog.smithi172:Node is ready 2023-12-10T00:01:42.542 DEBUG:teuthology.orchestra.run.smithi172:> hostname 2023-12-10T00:01:42.556 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com 2023-12-10T00:01:42.556 DEBUG:teuthology.orchestra.run.smithi172:> grep smithi172.front.sepia.ceph.com /etc/hosts 2023-12-10T00:01:42.611 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2023-12-10T00:01:42.612 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172 2023-12-10T00:01:42.721 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi172.front.sepia.ceph.com/smithi172/g' /etc/hosts 2023-12-10T00:01:42.748 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/172.21.15.172/172.21.15.172/g' /etc/hosts 2023-12-10T00:01:42.816 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi172.front.sepia.ceph.com/smithi172/g' /etc/hostname 2023-12-10T00:01:42.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172 2023-12-10T00:01:42.949 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-12-10T00:01:43.003 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2023-12-10T00:01:43.003 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2023-12-10T00:01:43.003 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2023-12-10T00:01:43.003 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2023-12-10T00:01:43.003 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2023-12-10T00:01:43.003 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2023-12-10T00:01:43.004 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T00:01:43.004 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-12-10T00:01:43.004 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2023-12-10T00:01:43.004 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T00:01:43.004 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2023-12-10T00:01:43.004 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T00:01:43.004 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T00:01:43.004 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T00:01:43.004 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2023-12-10T00:01:43.005 DEBUG:teuthology.parallel:result is None 2023-12-10T00:01:43.228 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com 2023-12-10T00:01:43.229 INFO:teuthology.lock.ops:Checking smithi040.front.sepia.ceph.com 2023-12-10T00:01:43.229 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2023-12-10T00:01:43.229 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-10T00:01:43.257 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server 2023-12-10T00:01:43.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-12-10T00:01:43.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:01:43.352 DEBUG:teuthology.orchestra.run.smithi022:> true 2023-12-10T00:01:43.420 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-12-10T00:01:43.420 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2023-12-10T00:01:43.475 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2023-12-10T00:01:43.475 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2023-12-10T00:01:43.530 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2023-12-10T00:01:43.531 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T00:01:43.531 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2023-12-10T00:01:43.531 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T00:01:43.531 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T00:01:43.531 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T00:01:43.531 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2023-12-10T00:01:43.547 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: set os type and version on server 2023-12-10T00:01:43.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-12-10T00:01:43.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:01:43.641 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-12-10T00:01:43.708 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-12-10T00:01:43.708 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2023-12-10T00:01:43.763 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2023-12-10T00:01:43.763 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2023-12-10T00:01:43.817 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2023-12-10T00:01:43.817 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2023-12-10T00:01:43.817 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2023-12-10T00:01:43.817 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2023-12-10T00:01:43.817 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2023-12-10T00:01:43.817 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2023-12-10T00:01:43.818 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T00:01:43.818 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2023-12-10T00:01:43.818 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2023-12-10T00:01:43.818 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T00:01:43.818 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2023-12-10T00:01:43.818 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T00:01:43.818 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T00:01:43.818 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T00:01:43.819 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2023-12-10T00:01:43.839 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2023-12-10T00:01:43.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-12-10T00:01:43.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T00:01:43.920 DEBUG:teuthology.orchestra.run.smithi172:> true 2023-12-10T00:01:43.989 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-12-10T00:01:43.989 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2023-12-10T00:01:44.045 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2023-12-10T00:01:44.045 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-10T00:01:44.101 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2023-12-10T00:01:44.102 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-10T00:01:44.102 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-10T00:01:44.102 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-10T00:01:44.102 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-12-10T00:01:44.122 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T00:01:44.299 INFO:teuthology.dispatcher.supervisor:Running job 7485459 2023-12-10T00:01:44.299 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_dis@teuthology --archive /home/teuthworker/archive/dis-2023-12-09_23:55:51-rbd-wip-dis-testing-distro-default-smithi/7485459 --name dis-2023-12-09_23:55:51-rbd-wip-dis-testing-distro-default-smithi --description rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} conf/{disable-pool-app} data-pool/none extra-conf/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} workloads/c_api_tests_with_journaling} -- /home/teuthworker/archive/dis-2023-12-09_23:55:51-rbd-wip-dis-testing-distro-default-smithi/7485459/orig.config.yaml 2023-12-10T00:01:44.302 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dis-2023-12-09_23:55:51-rbd-wip-dis-testing-distro-default-smithi/7485459 2023-12-10T00:01:44.302 INFO:teuthology.dispatcher.supervisor:Job PID: 1070101 2023-12-10T00:01:44.302 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-10T00:03:44.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T00:05:44.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T00:07:44.442 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T00:09:44.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T00:11:44.587 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T00:11:44.700 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-10T00:11:44.940 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi022 but it was locked by another job: /home/teuthworker/archive/dis-2023-12-09_23:57:15-rbd-wip-dis-testing-distro-default-smithi/7485523 2023-12-10T00:11:44.940 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi040 but it was locked by another job: /home/teuthworker/archive/dis-2023-12-09_23:57:15-rbd-wip-dis-testing-distro-default-smithi/7485523 2023-12-10T00:11:44.940 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-10T00:11:44.941 INFO:teuthology.nuke:Checking targets against current locks 2023-12-10T00:11:44.968 INFO:teuthology.nuke:Not nuking smithi022.front.sepia.ceph.com because description doesn't match: dis-2023-12-09_23:55:51-rbd-wip-dis-testing-distro-default-smithi != /home/teuthworker/archive/dis-2023-12-09_23:57:15-rbd-wip-dis-testing-distro-default-smithi/7485523 2023-12-10T00:11:45.021 INFO:teuthology.nuke:Not nuking smithi040.front.sepia.ceph.com because description doesn't match: dis-2023-12-09_23:55:51-rbd-wip-dis-testing-distro-default-smithi != /home/teuthworker/archive/dis-2023-12-09_23:57:15-rbd-wip-dis-testing-distro-default-smithi/7485523 2023-12-10T00:11:45.082 DEBUG:teuthology.nuke:shortname: smithi172 2023-12-10T00:11:45.083 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-10T00:11:45.110 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-12-09_23:55:51-rbd-wip-dis-testing-distro-default-smithi/7485459', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-09 23:56:44.483195', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNDEmRkziXZovGdH/JAvBO5LKZj9Hw9smFGMYD9si+6/KJDv9tHXWaETg5ZNlgB844m+LSEZuinV4F/c34tZJuo='} 2023-12-10T00:11:45.327 INFO:teuthology.orchestra.console.smithi172:Power off 2023-12-10T00:11:45.327 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T00:11:45.441 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2023-12-10T00:11:45.442 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T00:11:45.544 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-12-10T00:11:45.645 INFO:teuthology.orchestra.console.smithi172:Power off completed 2023-12-10T00:11:45.801 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com 2023-12-10T00:11:45.810 ERROR:teuthology.nuke:Could not nuke the following targets: targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGYWSy/lOeH6sOlq2OZOVjQfBABEWTAZiv5P5epCdW2k72IPq1dxSottnP8OCOr9scVYvA3ylJkW5p8J3QOv/Fk= smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCc7yC8vU+ZcZR94iu298OdAvTLNSDnNe5aVnp6x6AixWRs2Ex1Wm/2pFfhJRB9vzvnCL7wX1ElhiFwwDTHGuRI=