2024-02-06T23:24:06.386 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-06T23:24:06.387 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:24:06.563 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff5b55d2c10>, signals=[15]) 2024-02-06T23:24:06.563 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging 2024-02-06T23:24:06.563 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server 2024-02-06T23:24:06.563 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-02-06T23:24:06.583 INFO:teuthology.lock.ops:Start node 'smithi070.front.sepia.ceph.com' reimaging 2024-02-06T23:24:06.583 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: reset os type and version on server 2024-02-06T23:24:06.584 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-02-06T23:24:06.607 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging 2024-02-06T23:24:06.607 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server 2024-02-06T23:24:06.608 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-02-06T23:24:06.696 DEBUG:teuthology.provision.fog.smithi002:Requesting image smithi_centos_9.stream (ID 10) 2024-02-06T23:24:06.734 DEBUG:teuthology.provision.fog.smithi070:Requesting image smithi_centos_9.stream (ID 10) 2024-02-06T23:24:06.741 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_centos_9.stream (ID 10) 2024-02-06T23:24:06.745 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 9.stream 2024-02-06T23:24:06.774 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of centos 9.stream 2024-02-06T23:24:06.797 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 9.stream 2024-02-06T23:24:06.970 INFO:teuthology.orchestra.console.smithi002:Power off 2024-02-06T23:24:06.971 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-06T23:24:06.994 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2024-02-06T23:24:06.994 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:07.007 INFO:teuthology.orchestra.console.smithi070:Power off 2024-02-06T23:24:07.008 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-06T23:24:07.020 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-02-06T23:24:07.029 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2024-02-06T23:24:07.029 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:07.048 INFO:teuthology.orchestra.console.smithi164:Power off 2024-02-06T23:24:07.048 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-06T23:24:07.056 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2024-02-06T23:24:07.069 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-02-06T23:24:07.070 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:07.093 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-02-06T23:24:07.121 INFO:teuthology.orchestra.console.smithi002:Power off completed 2024-02-06T23:24:07.157 INFO:teuthology.orchestra.console.smithi070:Power off completed 2024-02-06T23:24:07.194 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-02-06T23:24:07.223 INFO:teuthology.orchestra.console.smithi002:Power on 2024-02-06T23:24:07.223 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-06T23:24:07.245 DEBUG:teuthology.orchestra.console.smithi002:power on output: Chassis Power Control: Up/On 2024-02-06T23:24:07.246 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:07.258 INFO:teuthology.orchestra.console.smithi070:Power on 2024-02-06T23:24:07.259 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-06T23:24:07.267 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-02-06T23:24:07.281 DEBUG:teuthology.orchestra.console.smithi070:power on output: Chassis Power Control: Up/On 2024-02-06T23:24:07.282 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:07.294 INFO:teuthology.orchestra.console.smithi164:Power on 2024-02-06T23:24:07.294 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-06T23:24:07.303 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2024-02-06T23:24:07.315 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On 2024-02-06T23:24:07.315 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:07.335 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-02-06T23:24:11.272 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:11.304 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:11.337 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:11.382 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-02-06T23:24:11.413 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-02-06T23:24:11.446 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-02-06T23:24:11.514 INFO:teuthology.orchestra.console.smithi070:Power on completed 2024-02-06T23:24:11.615 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish 2024-02-06T23:24:15.386 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:15.447 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-06T23:24:15.497 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-02-06T23:24:15.555 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-02-06T23:24:15.599 INFO:teuthology.orchestra.console.smithi002:Power on completed 2024-02-06T23:24:15.656 INFO:teuthology.orchestra.console.smithi164:Power on completed 2024-02-06T23:24:15.700 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2024-02-06T23:24:15.757 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2024-02-06T23:26:31.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:26:34.624 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-02-06T23:26:40.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:26:42.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:26:43.708 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-02-06T23:26:45.692 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-02-06T23:26:49.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:26:51.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:26:52.764 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-02-06T23:26:54.780 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-02-06T23:26:58.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:00.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:01.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:01.820 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-02-06T23:27:03.836 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-02-06T23:27:04.764 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-02-06T23:27:07.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:09.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:10.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:10.880 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-02-06T23:27:12.892 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-02-06T23:27:13.820 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-02-06T23:27:16.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:18.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:19.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:19.968 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-02-06T23:27:21.948 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-02-06T23:27:22.876 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-02-06T23:27:25.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:27.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:28.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:31.008 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-02-06T23:27:31.932 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-02-06T23:27:37.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:37.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:40.092 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-02-06T23:27:40.988 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-02-06T23:27:46.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:46.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:27:50.044 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-02-06T23:27:56.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:28:26.004 WARNING:teuthology.provision.fog.smithi002:timed out 2024-02-06T23:28:32.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:28:32.275 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-06T23:28:32.276 DEBUG:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-06T23:28:32.620 INFO:teuthology.provision.fog.smithi002:Node is ready 2024-02-06T23:28:32.620 DEBUG:teuthology.orchestra.run.smithi002:> hostname 2024-02-06T23:28:32.635 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2024-02-06T23:28:32.635 DEBUG:teuthology.orchestra.run.smithi002:> grep smithi002.front.sepia.ceph.com /etc/hosts 2024-02-06T23:28:32.690 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2024-02-06T23:28:32.691 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2024-02-06T23:28:32.804 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hosts 2024-02-06T23:28:32.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/172.21.15.2/172.21.15.2/g' /etc/hosts 2024-02-06T23:28:32.908 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hostname 2024-02-06T23:28:32.979 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2024-02-06T23:28:33.049 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-06T23:28:33.104 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-02-06T23:28:33.105 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-06T23:28:33.105 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-06T23:28:33.105 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-06T23:28:33.105 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2024-02-06T23:28:33.107 DEBUG:teuthology.parallel:result is None 2024-02-06T23:28:46.108 WARNING:teuthology.provision.fog.smithi070:timed out 2024-02-06T23:28:52.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:28:52.353 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-06T23:28:52.353 DEBUG:teuthology.orchestra.run.smithi070:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-06T23:28:52.669 INFO:teuthology.provision.fog.smithi070:Node is ready 2024-02-06T23:28:52.670 DEBUG:teuthology.orchestra.run.smithi070:> hostname 2024-02-06T23:28:52.684 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070.front.sepia.ceph.com 2024-02-06T23:28:52.684 DEBUG:teuthology.orchestra.run.smithi070:> grep smithi070.front.sepia.ceph.com /etc/hosts 2024-02-06T23:28:52.742 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70 smithi070.front.sepia.ceph.com smithi070 2024-02-06T23:28:52.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname smithi070 2024-02-06T23:28:52.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/smithi070.front.sepia.ceph.com/smithi070/g' /etc/hosts 2024-02-06T23:28:52.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/172.21.15.70/172.21.15.70/g' /etc/hosts 2024-02-06T23:28:52.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/smithi070.front.sepia.ceph.com/smithi070/g' /etc/hostname 2024-02-06T23:28:53.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname smithi070 2024-02-06T23:28:53.105 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-02-06T23:28:53.163 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-02-06T23:28:53.163 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-02-06T23:28:53.163 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-02-06T23:28:53.163 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-02-06T23:28:53.163 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-06T23:28:53.164 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-06T23:28:53.164 INFO:teuthology.provision.fog.smithi070:Deploy complete! 2024-02-06T23:28:53.165 DEBUG:teuthology.parallel:result is None 2024-02-06T23:28:56.055 WARNING:teuthology.provision.fog.smithi164:timed out 2024-02-06T23:29:02.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:29:02.303 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-06T23:29:02.303 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-06T23:29:02.656 INFO:teuthology.provision.fog.smithi164:Node is ready 2024-02-06T23:29:02.657 DEBUG:teuthology.orchestra.run.smithi164:> hostname 2024-02-06T23:29:02.672 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2024-02-06T23:29:02.672 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164.front.sepia.ceph.com /etc/hosts 2024-02-06T23:29:02.729 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2024-02-06T23:29:02.729 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164 2024-02-06T23:29:02.838 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hosts 2024-02-06T23:29:02.866 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts 2024-02-06T23:29:02.935 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hostname 2024-02-06T23:29:03.003 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164 2024-02-06T23:29:03.069 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-06T23:29:03.125 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-02-06T23:29:03.126 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-06T23:29:03.126 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-06T23:29:03.126 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-06T23:29:03.126 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2024-02-06T23:29:03.126 DEBUG:teuthology.parallel:result is None 2024-02-06T23:29:03.351 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2024-02-06T23:29:03.352 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-06T23:29:03.376 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com 2024-02-06T23:29:03.376 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-06T23:29:03.392 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2024-02-06T23:29:03.392 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-06T23:29:03.410 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2024-02-06T23:29:03.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-02-06T23:29:03.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:29:03.515 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-02-06T23:29:03.694 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-02-06T23:29:03.695 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-02-06T23:29:03.749 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-02-06T23:29:03.750 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-02-06T23:29:03.804 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-02-06T23:29:03.804 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-06T23:29:03.805 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-06T23:29:03.806 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-02-06T23:29:03.826 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server 2024-02-06T23:29:03.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-02-06T23:29:03.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:29:03.914 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-02-06T23:29:04.082 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-02-06T23:29:04.082 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-02-06T23:29:04.137 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-02-06T23:29:04.137 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-02-06T23:29:04.191 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-02-06T23:29:04.191 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-02-06T23:29:04.191 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-02-06T23:29:04.191 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-02-06T23:29:04.191 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-06T23:29:04.192 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-06T23:29:04.192 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-02-06T23:29:04.217 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2024-02-06T23:29:04.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-02-06T23:29:04.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-06T23:29:04.309 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-02-06T23:29:04.384 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-02-06T23:29:04.384 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-02-06T23:29:04.440 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-02-06T23:29:04.441 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-02-06T23:29:04.496 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-02-06T23:29:04.496 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-02-06T23:29:04.496 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-02-06T23:29:04.496 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-06T23:29:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-06T23:29:04.498 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-02-06T23:29:04.517 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:29:04.611 INFO:teuthology.dispatcher.supervisor:Running job 7548462 2024-02-06T23:29:04.611 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-02-06_01:32:07-rados-wip-yuri6-testing-2024-02-05-1523-distro-default-smithi/7548462 --name yuriw-2024-02-06_01:32:07-rados-wip-yuri6-testing-2024-02-05-1523-distro-default-smithi --description rados/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_set_mon_crush_locations} -- /home/teuthworker/archive/yuriw-2024-02-06_01:32:07-rados-wip-yuri6-testing-2024-02-05-1523-distro-default-smithi/7548462/orig.config.yaml 2024-02-06T23:29:04.616 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-02-06_01:32:07-rados-wip-yuri6-testing-2024-02-05-1523-distro-default-smithi/7548462 2024-02-06T23:29:04.616 INFO:teuthology.dispatcher.supervisor:Job PID: 123298 2024-02-06T23:29:04.616 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-06T23:31:04.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:33:04.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:35:05.045 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:37:05.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:39:05.369 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:41:05.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:43:05.717 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-06T23:43:05.786 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1