2024-03-26T01:20:13.648 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T01:20:13.648 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:20:13.831 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f949f5edc10>, signals=[15]) 2024-03-26T01:20:13.831 INFO:teuthology.lock.ops:Start node 'smithi060.front.sepia.ceph.com' reimaging 2024-03-26T01:20:13.832 INFO:teuthology.lock.ops:Updating [smithi060.front.sepia.ceph.com]: reset os type and version on server 2024-03-26T01:20:13.832 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-03-26T01:20:13.852 INFO:teuthology.lock.ops:Start node 'smithi086.front.sepia.ceph.com' reimaging 2024-03-26T01:20:13.852 INFO:teuthology.lock.ops:Updating [smithi086.front.sepia.ceph.com]: reset os type and version on server 2024-03-26T01:20:13.852 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-03-26T01:20:13.868 INFO:teuthology.lock.ops:Start node 'smithi129.front.sepia.ceph.com' reimaging 2024-03-26T01:20:13.868 INFO:teuthology.lock.ops:Updating [smithi129.front.sepia.ceph.com]: reset os type and version on server 2024-03-26T01:20:13.868 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-03-26T01:20:13.963 DEBUG:teuthology.provision.fog.smithi060:Requesting image smithi_centos_8.stream (ID 3) 2024-03-26T01:20:13.975 DEBUG:teuthology.provision.fog.smithi086:Requesting image smithi_centos_8.stream (ID 3) 2024-03-26T01:20:14.001 DEBUG:teuthology.provision.fog.smithi129:Requesting image smithi_centos_8.stream (ID 3) 2024-03-26T01:20:14.013 INFO:teuthology.provision.fog.smithi086:Scheduling deploy of centos 8.stream 2024-03-26T01:20:14.016 INFO:teuthology.provision.fog.smithi060:Scheduling deploy of centos 8.stream 2024-03-26T01:20:14.046 INFO:teuthology.provision.fog.smithi129:Scheduling deploy of centos 8.stream 2024-03-26T01:20:14.219 INFO:teuthology.orchestra.console.smithi060:Power off 2024-03-26T01:20:14.219 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T01:20:14.234 INFO:teuthology.orchestra.console.smithi086:Power off 2024-03-26T01:20:14.234 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T01:20:14.245 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2024-03-26T01:20:14.246 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:14.255 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-03-26T01:20:14.255 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:14.272 INFO:teuthology.orchestra.console.smithi129:Power off 2024-03-26T01:20:14.272 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T01:20:14.280 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-03-26T01:20:14.281 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-03-26T01:20:14.295 DEBUG:teuthology.orchestra.console.smithi129:power off output: Chassis Power Control: Down/Off 2024-03-26T01:20:14.295 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:14.320 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2024-03-26T01:20:14.381 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-03-26T01:20:14.482 INFO:teuthology.orchestra.console.smithi086:Power on 2024-03-26T01:20:14.482 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-26T01:20:14.506 DEBUG:teuthology.orchestra.console.smithi086:power on output: Chassis Power Control: Up/On 2024-03-26T01:20:14.506 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:14.527 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-03-26T01:20:18.286 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:18.321 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:18.398 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-03-26T01:20:18.432 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2024-03-26T01:20:18.528 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:18.637 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-03-26T01:20:22.403 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:22.433 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:22.512 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-03-26T01:20:22.542 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2024-03-26T01:20:22.638 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:22.646 INFO:teuthology.orchestra.console.smithi129:Power off completed 2024-03-26T01:20:22.747 INFO:teuthology.orchestra.console.smithi129:Power on 2024-03-26T01:20:22.747 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-26T01:20:22.757 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-03-26T01:20:22.770 DEBUG:teuthology.orchestra.console.smithi129:power on output: Chassis Power Control: Up/On 2024-03-26T01:20:22.773 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:22.795 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2024-03-26T01:20:22.858 INFO:teuthology.orchestra.console.smithi086:Power on completed 2024-03-26T01:20:22.959 INFO:teuthology.provision.fog.smithi086:Waiting for deploy to finish 2024-03-26T01:20:26.514 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:26.623 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2024-03-26T01:20:26.724 INFO:teuthology.orchestra.console.smithi060:Power off completed 2024-03-26T01:20:26.797 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:26.826 INFO:teuthology.orchestra.console.smithi060:Power on 2024-03-26T01:20:26.826 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-26T01:20:26.847 DEBUG:teuthology.orchestra.console.smithi060:power on output: Chassis Power Control: Up/On 2024-03-26T01:20:26.848 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:26.873 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2024-03-26T01:20:26.906 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2024-03-26T01:20:30.878 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:30.907 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T01:20:30.989 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-03-26T01:20:31.018 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2024-03-26T01:20:31.090 INFO:teuthology.orchestra.console.smithi060:Power on completed 2024-03-26T01:20:31.119 INFO:teuthology.orchestra.console.smithi129:Power on completed 2024-03-26T01:20:31.192 INFO:teuthology.provision.fog.smithi060:Waiting for deploy to finish 2024-03-26T01:20:31.220 INFO:teuthology.provision.fog.smithi129:Waiting for deploy to finish 2024-03-26T01:22:53.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:22:56.862 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:23:02.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:02.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:05.118 WARNING:teuthology.provision.fog.smithi060:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-03-26T01:23:05.918 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:23:11.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:11.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:14.174 WARNING:teuthology.provision.fog.smithi060:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-03-26T01:23:14.974 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:23:17.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:20.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:20.190 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-03-26T01:23:20.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:23.230 WARNING:teuthology.provision.fog.smithi060:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-03-26T01:23:24.030 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:23:26.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:29.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:29.246 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-03-26T01:23:30.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:32.286 WARNING:teuthology.provision.fog.smithi060:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-03-26T01:23:33.086 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:23:35.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:38.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:38.306 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-03-26T01:23:39.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:41.342 WARNING:teuthology.provision.fog.smithi060:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-03-26T01:23:42.146 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:23:44.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:47.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:47.394 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-03-26T01:23:48.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:51.198 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:23:53.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:23:56.446 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-03-26T01:23:57.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:24:02.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:24:05.502 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-03-26T01:24:11.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:24:15.518 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:24:21.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:24:24.574 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-26T01:24:30.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:24:31.700 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-26T01:24:31.701 DEBUG:teuthology.orchestra.run.smithi086:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-26T01:24:47.349 WARNING:teuthology.provision.fog.smithi060:timed out 2024-03-26T01:24:53.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:24:53.599 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-26T01:24:53.600 DEBUG:teuthology.orchestra.run.smithi060:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-26T01:24:53.872 INFO:teuthology.provision.fog.smithi060:Node is ready 2024-03-26T01:24:53.872 DEBUG:teuthology.orchestra.run.smithi060:> hostname 2024-03-26T01:24:53.892 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060.front.sepia.ceph.com 2024-03-26T01:24:53.892 DEBUG:teuthology.orchestra.run.smithi060:> grep smithi060.front.sepia.ceph.com /etc/hosts 2024-03-26T01:24:53.952 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60 smithi060.front.sepia.ceph.com smithi060 2024-03-26T01:24:53.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname smithi060 2024-03-26T01:24:54.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i -e 's/smithi060.front.sepia.ceph.com/smithi060/g' /etc/hosts 2024-03-26T01:24:54.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i -e 's/172.21.15.60/172.21.15.60/g' /etc/hosts 2024-03-26T01:24:54.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i -e 's/smithi060.front.sepia.ceph.com/smithi060/g' /etc/hostname 2024-03-26T01:24:54.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname smithi060 2024-03-26T01:24:54.302 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="8" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="8" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el8" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-26T01:24:54.362 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-03-26T01:24:54.363 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T01:24:54.363 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-26T01:24:54.363 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T01:24:54.364 INFO:teuthology.provision.fog.smithi060:Deploy complete! 2024-03-26T01:24:54.365 DEBUG:teuthology.parallel:result is None 2024-03-26T01:24:56.936 INFO:teuthology.provision.fog.smithi086:Node is ready 2024-03-26T01:24:56.936 DEBUG:teuthology.orchestra.run.smithi086:> hostname 2024-03-26T01:24:56.961 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086.front.sepia.ceph.com 2024-03-26T01:24:56.962 DEBUG:teuthology.orchestra.run.smithi086:> grep smithi086.front.sepia.ceph.com /etc/hosts 2024-03-26T01:24:57.024 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86 smithi086.front.sepia.ceph.com smithi086 2024-03-26T01:24:57.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname smithi086 2024-03-26T01:24:57.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/smithi086.front.sepia.ceph.com/smithi086/g' /etc/hosts 2024-03-26T01:24:57.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/172.21.15.86/172.21.15.86/g' /etc/hosts 2024-03-26T01:24:57.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/smithi086.front.sepia.ceph.com/smithi086/g' /etc/hostname 2024-03-26T01:24:57.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname smithi086 2024-03-26T01:24:57.391 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-03-26T01:24:57.449 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-03-26T01:24:57.449 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8" 2024-03-26T01:24:57.449 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-03-26T01:24:57.449 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-03-26T01:24:57.449 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8" 2024-03-26T01:24:57.450 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2024-03-26T01:24:57.450 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-26T01:24:57.450 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-03-26T01:24:57.450 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-26T01:24:57.450 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-03-26T01:24:57.450 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T01:24:57.450 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-26T01:24:57.450 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T01:24:57.450 INFO:teuthology.provision.fog.smithi086:Deploy complete! 2024-03-26T01:24:57.451 DEBUG:teuthology.parallel:result is None 2024-03-26T01:25:11.522 WARNING:teuthology.provision.fog.smithi129:timed out 2024-03-26T01:25:17.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:25:17.742 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-26T01:25:17.742 DEBUG:teuthology.orchestra.run.smithi129:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-26T01:25:18.004 INFO:teuthology.provision.fog.smithi129:Node is ready 2024-03-26T01:25:18.004 DEBUG:teuthology.orchestra.run.smithi129:> hostname 2024-03-26T01:25:18.024 INFO:teuthology.orchestra.run.smithi129.stdout:smithi129.front.sepia.ceph.com 2024-03-26T01:25:18.025 DEBUG:teuthology.orchestra.run.smithi129:> grep smithi129.front.sepia.ceph.com /etc/hosts 2024-03-26T01:25:18.086 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129 smithi129.front.sepia.ceph.com smithi129 2024-03-26T01:25:18.086 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname smithi129 2024-03-26T01:25:18.192 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i -e 's/smithi129.front.sepia.ceph.com/smithi129/g' /etc/hosts 2024-03-26T01:25:18.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i -e 's/172.21.15.129/172.21.15.129/g' /etc/hosts 2024-03-26T01:25:18.296 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i -e 's/smithi129.front.sepia.ceph.com/smithi129/g' /etc/hostname 2024-03-26T01:25:18.368 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname smithi129 2024-03-26T01:25:18.437 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-03-26T01:25:18.496 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-03-26T01:25:18.496 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="8" 2024-03-26T01:25:18.496 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-03-26T01:25:18.496 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-03-26T01:25:18.496 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="8" 2024-03-26T01:25:18.496 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el8" 2024-03-26T01:25:18.496 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-26T01:25:18.496 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-03-26T01:25:18.497 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-26T01:25:18.497 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-03-26T01:25:18.497 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T01:25:18.497 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-26T01:25:18.497 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T01:25:18.497 INFO:teuthology.provision.fog.smithi129:Deploy complete! 2024-03-26T01:25:18.498 DEBUG:teuthology.parallel:result is None 2024-03-26T01:25:18.720 INFO:teuthology.lock.ops:Checking smithi060.front.sepia.ceph.com 2024-03-26T01:25:18.720 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-26T01:25:18.755 INFO:teuthology.lock.ops:Checking smithi086.front.sepia.ceph.com 2024-03-26T01:25:18.756 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-26T01:25:18.788 INFO:teuthology.lock.ops:Checking smithi129.front.sepia.ceph.com 2024-03-26T01:25:18.788 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-26T01:25:18.806 INFO:teuthology.lock.ops:Updating [smithi060.front.sepia.ceph.com]: set os type and version on server 2024-03-26T01:25:18.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-03-26T01:25:18.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:25:18.882 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-03-26T01:25:19.023 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-03-26T01:25:19.023 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-03-26T01:25:19.084 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-03-26T01:25:19.085 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-03-26T01:25:19.145 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="8" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="8" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el8" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-26T01:25:19.146 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T01:25:19.147 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-03-26T01:25:19.163 INFO:teuthology.lock.ops:Updating [smithi086.front.sepia.ceph.com]: set os type and version on server 2024-03-26T01:25:19.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-26T01:25:19.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:25:19.239 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-26T01:25:19.331 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-26T01:25:19.331 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-03-26T01:25:19.390 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-03-26T01:25:19.391 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-26T01:25:19.449 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-03-26T01:25:19.450 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T01:25:19.450 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-26T01:25:19.450 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T01:25:19.450 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-03-26T01:25:19.481 INFO:teuthology.lock.ops:Updating [smithi129.front.sepia.ceph.com]: set os type and version on server 2024-03-26T01:25:19.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-03-26T01:25:19.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T01:25:19.553 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-03-26T01:25:19.623 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-03-26T01:25:19.623 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-03-26T01:25:19.684 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-03-26T01:25:19.684 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-03-26T01:25:19.745 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-03-26T01:25:19.745 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="8" 2024-03-26T01:25:19.745 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-03-26T01:25:19.745 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="8" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el8" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-26T01:25:19.746 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T01:25:19.747 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-03-26T01:25:19.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:25:19.872 INFO:teuthology.dispatcher.supervisor:Running job 7621889 2024-03-26T01:25:19.872 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi/7621889 --name yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi --description rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} -- /home/teuthworker/archive/yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi/7621889/orig.config.yaml 2024-03-26T01:25:19.875 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi/7621889 2024-03-26T01:25:19.875 INFO:teuthology.dispatcher.supervisor:Job PID: 3466196 2024-03-26T01:25:19.875 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-26T01:27:19.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:29:20.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:31:20.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:33:20.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:35:20.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:37:20.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:39:21.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:41:21.199 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:43:21.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:45:21.539 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:47:21.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:49:21.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:51:22.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:53:22.217 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:55:22.383 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:57:22.547 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T01:59:22.720 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:01:22.895 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:03:23.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:05:23.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:07:23.413 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:09:23.571 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:11:23.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:13:23.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:15:24.175 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:17:24.353 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:19:24.500 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:21:24.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:23:24.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:25:24.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:27:25.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:29:25.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:31:25.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:33:25.657 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:35:25.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:37:25.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:39:26.135 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:41:26.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:43:26.412 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:45:26.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:47:26.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T02:47:26.860 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1