2024-02-08T16:23:58.910 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-08T16:23:58.910 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:23:59.114 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f456f86fc10>, signals=[15]) 2024-02-08T16:23:59.115 INFO:teuthology.lock.ops:Start node 'smithi039.front.sepia.ceph.com' reimaging 2024-02-08T16:23:59.115 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: reset os type and version on server 2024-02-08T16:23:59.115 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-02-08T16:23:59.141 INFO:teuthology.lock.ops:Start node 'smithi152.front.sepia.ceph.com' reimaging 2024-02-08T16:23:59.141 INFO:teuthology.lock.ops:Updating [smithi152.front.sepia.ceph.com]: reset os type and version on server 2024-02-08T16:23:59.141 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-02-08T16:23:59.165 INFO:teuthology.lock.ops:Start node 'smithi162.front.sepia.ceph.com' reimaging 2024-02-08T16:23:59.166 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: reset os type and version on server 2024-02-08T16:23:59.166 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-02-08T16:23:59.266 DEBUG:teuthology.provision.fog.smithi039:Requesting image smithi_centos_9.stream (ID 10) 2024-02-08T16:23:59.271 DEBUG:teuthology.provision.fog.smithi152:Requesting image smithi_centos_9.stream (ID 10) 2024-02-08T16:23:59.299 DEBUG:teuthology.provision.fog.smithi162:Requesting image smithi_centos_9.stream (ID 10) 2024-02-08T16:23:59.311 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of centos 9.stream 2024-02-08T16:23:59.314 INFO:teuthology.provision.fog.smithi152:Scheduling deploy of centos 9.stream 2024-02-08T16:23:59.338 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of centos 9.stream 2024-02-08T16:23:59.526 INFO:teuthology.orchestra.console.smithi152:Power off 2024-02-08T16:23:59.526 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-08T16:23:59.542 INFO:teuthology.orchestra.console.smithi039:Power off 2024-02-08T16:23:59.543 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-08T16:23:59.551 DEBUG:teuthology.orchestra.console.smithi152:power off output: Chassis Power Control: Down/Off 2024-02-08T16:23:59.552 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:23:59.567 INFO:teuthology.orchestra.console.smithi162:Power off 2024-02-08T16:23:59.567 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-08T16:23:59.576 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2024-02-08T16:23:59.577 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:23:59.584 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-02-08T16:23:59.591 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2024-02-08T16:23:59.591 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:23:59.600 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-02-08T16:23:59.613 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2024-02-08T16:24:03.588 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:03.600 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:03.614 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:03.700 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-02-08T16:24:03.710 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-02-08T16:24:03.723 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2024-02-08T16:24:07.704 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:07.711 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:07.723 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:07.814 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-02-08T16:24:07.819 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-02-08T16:24:07.832 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2024-02-08T16:24:07.933 INFO:teuthology.orchestra.console.smithi162:Power off completed 2024-02-08T16:24:08.034 INFO:teuthology.orchestra.console.smithi162:Power on 2024-02-08T16:24:08.034 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-08T16:24:08.058 DEBUG:teuthology.orchestra.console.smithi162:power on output: Chassis Power Control: Up/On 2024-02-08T16:24:08.058 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:08.085 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2024-02-08T16:24:11.819 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:11.827 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:11.929 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is off 2024-02-08T16:24:11.936 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-02-08T16:24:12.031 INFO:teuthology.orchestra.console.smithi152:Power off completed 2024-02-08T16:24:12.037 INFO:teuthology.orchestra.console.smithi039:Power off completed 2024-02-08T16:24:12.086 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:12.132 INFO:teuthology.orchestra.console.smithi152:Power on 2024-02-08T16:24:12.133 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-08T16:24:12.140 INFO:teuthology.orchestra.console.smithi039:Power on 2024-02-08T16:24:12.140 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-08T16:24:12.153 DEBUG:teuthology.orchestra.console.smithi152:power on output: Chassis Power Control: Up/On 2024-02-08T16:24:12.154 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:12.165 DEBUG:teuthology.orchestra.console.smithi039:power on output: Chassis Power Control: Up/On 2024-02-08T16:24:12.165 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:12.174 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is off 2024-02-08T16:24:12.189 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-02-08T16:24:12.198 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2024-02-08T16:24:16.179 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:16.191 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:16.200 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T16:24:16.288 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-02-08T16:24:16.299 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-02-08T16:24:16.365 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2024-02-08T16:24:16.390 INFO:teuthology.orchestra.console.smithi152:Power on completed 2024-02-08T16:24:16.400 INFO:teuthology.orchestra.console.smithi039:Power on completed 2024-02-08T16:24:16.467 INFO:teuthology.orchestra.console.smithi162:Power on completed 2024-02-08T16:24:16.490 INFO:teuthology.provision.fog.smithi152:Waiting for deploy to finish 2024-02-08T16:24:16.502 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish 2024-02-08T16:24:16.568 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish 2024-02-08T16:27:02.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:02.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:05.500 WARNING:teuthology.provision.fog.smithi152:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-02-08T16:27:05.504 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-02-08T16:27:11.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:11.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:14.556 WARNING:teuthology.provision.fog.smithi152:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-02-08T16:27:14.588 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-02-08T16:27:17.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:20.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:20.576 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:27:20.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:23.616 WARNING:teuthology.provision.fog.smithi152:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-02-08T16:27:23.648 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-02-08T16:27:26.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:29.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:29.632 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:27:29.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:32.700 WARNING:teuthology.provision.fog.smithi152:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-02-08T16:27:32.701 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-02-08T16:27:35.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:38.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:38.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:38.716 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:27:41.756 WARNING:teuthology.provision.fog.smithi152:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-02-08T16:27:41.756 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-02-08T16:27:44.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:47.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:47.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:47.776 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:27:50.812 WARNING:teuthology.provision.fog.smithi152:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-02-08T16:27:50.844 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-02-08T16:27:53.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:56.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:56.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:27:56.860 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:27:59.872 WARNING:teuthology.provision.fog.smithi152:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-02-08T16:27:59.900 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-02-08T16:28:02.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:28:05.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:28:05.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:28:05.916 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:28:08.956 WARNING:teuthology.provision.fog.smithi152:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-02-08T16:28:11.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:28:14.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:28:14.972 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:28:20.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:28:24.028 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:28:30.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:28:33.180 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-02-08T16:28:39.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:29:05.930 WARNING:teuthology.provision.fog.smithi162:timed out 2024-02-08T16:29:11.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:29:12.237 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-08T16:29:12.238 DEBUG:teuthology.orchestra.run.smithi162:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-08T16:29:12.555 INFO:teuthology.provision.fog.smithi162:Node is ready 2024-02-08T16:29:12.555 DEBUG:teuthology.orchestra.run.smithi162:> hostname 2024-02-08T16:29:12.570 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com 2024-02-08T16:29:12.571 DEBUG:teuthology.orchestra.run.smithi162:> grep smithi162.front.sepia.ceph.com /etc/hosts 2024-02-08T16:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2024-02-08T16:29:12.627 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname smithi162 2024-02-08T16:29:12.739 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/smithi162.front.sepia.ceph.com/smithi162/g' /etc/hosts 2024-02-08T16:29:12.766 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/172.21.15.162/172.21.15.162/g' /etc/hosts 2024-02-08T16:29:12.833 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/smithi162.front.sepia.ceph.com/smithi162/g' /etc/hostname 2024-02-08T16:29:12.901 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname smithi162 2024-02-08T16:29:12.967 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-02-08T16:29:13.024 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-02-08T16:29:13.024 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-02-08T16:29:13.024 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-08T16:29:13.025 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-08T16:29:13.026 INFO:teuthology.provision.fog.smithi162:Deploy complete! 2024-02-08T16:29:13.028 DEBUG:teuthology.parallel:result is None 2024-02-08T16:29:14.964 WARNING:teuthology.provision.fog.smithi152:timed out 2024-02-08T16:29:20.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:29:21.220 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-08T16:29:21.220 DEBUG:teuthology.orchestra.run.smithi152:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-08T16:29:21.552 INFO:teuthology.provision.fog.smithi152:Node is ready 2024-02-08T16:29:21.552 DEBUG:teuthology.orchestra.run.smithi152:> hostname 2024-02-08T16:29:21.567 INFO:teuthology.orchestra.run.smithi152.stdout:smithi152.front.sepia.ceph.com 2024-02-08T16:29:21.567 DEBUG:teuthology.orchestra.run.smithi152:> grep smithi152.front.sepia.ceph.com /etc/hosts 2024-02-08T16:29:21.623 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152 smithi152.front.sepia.ceph.com smithi152 2024-02-08T16:29:21.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname smithi152 2024-02-08T16:29:21.750 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i -e 's/smithi152.front.sepia.ceph.com/smithi152/g' /etc/hosts 2024-02-08T16:29:21.779 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i -e 's/172.21.15.152/172.21.15.152/g' /etc/hosts 2024-02-08T16:29:21.847 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i -e 's/smithi152.front.sepia.ceph.com/smithi152/g' /etc/hostname 2024-02-08T16:29:21.915 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname smithi152 2024-02-08T16:29:21.983 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-02-08T16:29:22.037 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-02-08T16:29:22.039 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-02-08T16:29:22.039 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-02-08T16:29:22.039 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-08T16:29:22.040 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-08T16:29:22.040 INFO:teuthology.provision.fog.smithi152:Deploy complete! 2024-02-08T16:29:22.042 DEBUG:teuthology.parallel:result is None 2024-02-08T16:29:39.204 WARNING:teuthology.provision.fog.smithi039:timed out 2024-02-08T16:29:45.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:29:45.541 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-08T16:29:45.542 DEBUG:teuthology.orchestra.run.smithi039:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-08T16:29:45.849 INFO:teuthology.provision.fog.smithi039:Node is ready 2024-02-08T16:29:45.849 DEBUG:teuthology.orchestra.run.smithi039:> hostname 2024-02-08T16:29:45.904 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039.front.sepia.ceph.com 2024-02-08T16:29:45.905 DEBUG:teuthology.orchestra.run.smithi039:> grep smithi039.front.sepia.ceph.com /etc/hosts 2024-02-08T16:29:45.961 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39 smithi039.front.sepia.ceph.com smithi039 2024-02-08T16:29:45.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname smithi039 2024-02-08T16:29:46.059 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/smithi039.front.sepia.ceph.com/smithi039/g' /etc/hosts 2024-02-08T16:29:46.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/172.21.15.39/172.21.15.39/g' /etc/hosts 2024-02-08T16:29:46.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/smithi039.front.sepia.ceph.com/smithi039/g' /etc/hostname 2024-02-08T16:29:46.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname smithi039 2024-02-08T16:29:46.306 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-02-08T16:29:46.361 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-02-08T16:29:46.361 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-08T16:29:46.362 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-08T16:29:46.363 INFO:teuthology.provision.fog.smithi039:Deploy complete! 2024-02-08T16:29:46.364 DEBUG:teuthology.parallel:result is None 2024-02-08T16:29:46.641 INFO:teuthology.lock.ops:Checking smithi039.front.sepia.ceph.com 2024-02-08T16:29:46.641 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-08T16:29:46.662 INFO:teuthology.lock.ops:Checking smithi152.front.sepia.ceph.com 2024-02-08T16:29:46.662 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-08T16:29:46.678 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com 2024-02-08T16:29:46.678 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-08T16:29:46.696 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: set os type and version on server 2024-02-08T16:29:46.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-02-08T16:29:46.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:29:46.799 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-02-08T16:29:46.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-02-08T16:29:46.867 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-02-08T16:29:46.921 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-02-08T16:29:46.922 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-02-08T16:29:46.977 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-08T16:29:46.978 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-02-08T16:29:46.978 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-08T16:29:46.978 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-08T16:29:46.978 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-08T16:29:46.978 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-02-08T16:29:47.004 INFO:teuthology.lock.ops:Updating [smithi152.front.sepia.ceph.com]: set os type and version on server 2024-02-08T16:29:47.004 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-02-08T16:29:47.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:29:47.101 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-02-08T16:29:47.270 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-02-08T16:29:47.270 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-02-08T16:29:47.325 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-02-08T16:29:47.325 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-08T16:29:47.380 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-02-08T16:29:47.381 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-08T16:29:47.381 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-08T16:29:47.381 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-08T16:29:47.381 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-02-08T16:29:47.404 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server 2024-02-08T16:29:47.404 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-02-08T16:29:47.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T16:29:47.535 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-02-08T16:29:47.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-02-08T16:29:47.702 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-02-08T16:29:47.757 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-02-08T16:29:47.758 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-02-08T16:29:47.812 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-08T16:29:47.813 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-08T16:29:47.814 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-02-08T16:29:47.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:29:47.928 INFO:teuthology.dispatcher.supervisor:Running job 7551432 2024-02-08T16:29:47.928 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-02-08_00:19:25-rados-wip-yuri3-testing-2024-02-07-1233-distro-default-smithi/7551432 --name yuriw-2024-02-08_00:19:25-rados-wip-yuri3-testing-2024-02-07-1233-distro-default-smithi --description rados/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_set_mon_crush_locations} -- /home/teuthworker/archive/yuriw-2024-02-08_00:19:25-rados-wip-yuri3-testing-2024-02-07-1233-distro-default-smithi/7551432/orig.config.yaml 2024-02-08T16:29:47.933 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-02-08_00:19:25-rados-wip-yuri3-testing-2024-02-07-1233-distro-default-smithi/7551432 2024-02-08T16:29:47.933 INFO:teuthology.dispatcher.supervisor:Job PID: 1576247 2024-02-08T16:29:47.934 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-08T16:31:48.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:33:48.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:35:48.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:37:48.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:39:48.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:41:48.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:43:49.122 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:45:49.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:47:49.434 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:49:49.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:51:49.769 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T16:51:49.838 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1