2024-02-21T06:30:12.090 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55
2024-02-21T06:30:12.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:30:12.289 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fe4a29bff70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fe4a3a20c10>, signals=[15])
2024-02-21T06:30:12.289 INFO:teuthology.lock.ops:Start node 'smithi003.front.sepia.ceph.com' reimaging
2024-02-21T06:30:12.289 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: reset os type and version on server
2024-02-21T06:30:12.290 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2024-02-21T06:30:12.312 INFO:teuthology.lock.ops:Start node 'smithi038.front.sepia.ceph.com' reimaging
2024-02-21T06:30:12.313 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: reset os type and version on server
2024-02-21T06:30:12.313 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2024-02-21T06:30:12.346 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging
2024-02-21T06:30:12.346 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server
2024-02-21T06:30:12.346 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2024-02-21T06:30:12.428 DEBUG:teuthology.provision.fog.smithi003:Requesting image smithi_centos_9.stream (ID 10)
2024-02-21T06:30:12.457 DEBUG:teuthology.provision.fog.smithi038:Requesting image smithi_centos_9.stream (ID 10)
2024-02-21T06:30:12.474 DEBUG:teuthology.provision.fog.smithi121:Requesting image smithi_centos_9.stream (ID 10)
2024-02-21T06:30:12.491 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of centos 9.stream
2024-02-21T06:30:12.507 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of centos 9.stream
2024-02-21T06:30:12.516 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of centos 9.stream
2024-02-21T06:30:12.934 INFO:teuthology.orchestra.console.smithi121:Power off
2024-02-21T06:30:12.934 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-21T06:30:12.948 INFO:teuthology.orchestra.console.smithi038:Power off
2024-02-21T06:30:12.949 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-21T06:30:12.962 INFO:teuthology.orchestra.console.smithi003:Power off
2024-02-21T06:30:12.962 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-21T06:30:12.969 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off
2024-02-21T06:30:12.970 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:12.979 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off
2024-02-21T06:30:12.980 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:12.989 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off
2024-02-21T06:30:12.989 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:12.996 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off
2024-02-21T06:30:13.003 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on
2024-02-21T06:30:13.011 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2024-02-21T06:30:13.097 INFO:teuthology.orchestra.console.smithi121:Power off completed
2024-02-21T06:30:13.197 INFO:teuthology.orchestra.console.smithi121:Power on
2024-02-21T06:30:13.198 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-21T06:30:13.218 DEBUG:teuthology.orchestra.console.smithi121:power on output: Chassis Power Control: Up/On
2024-02-21T06:30:13.218 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:13.242 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off
2024-02-21T06:30:17.007 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:17.014 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:17.117 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on
2024-02-21T06:30:17.123 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2024-02-21T06:30:17.243 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:17.353 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on
2024-02-21T06:30:17.454 INFO:teuthology.orchestra.console.smithi121:Power on completed
2024-02-21T06:30:17.555 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish
2024-02-21T06:30:21.120 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:21.128 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:21.230 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on
2024-02-21T06:30:21.236 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2024-02-21T06:30:25.235 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:25.242 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:25.345 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2024-02-21T06:30:25.350 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2024-02-21T06:30:25.446 INFO:teuthology.orchestra.console.smithi038:Power off completed
2024-02-21T06:30:25.450 INFO:teuthology.orchestra.console.smithi003:Power off completed
2024-02-21T06:30:25.547 INFO:teuthology.orchestra.console.smithi038:Power on
2024-02-21T06:30:25.547 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-21T06:30:25.554 INFO:teuthology.orchestra.console.smithi003:Power on
2024-02-21T06:30:25.554 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-21T06:30:25.568 DEBUG:teuthology.orchestra.console.smithi038:power on output: Chassis Power Control: Up/On
2024-02-21T06:30:25.569 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:25.577 DEBUG:teuthology.orchestra.console.smithi003:power on output: Chassis Power Control: Up/On
2024-02-21T06:30:25.577 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:25.593 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2024-02-21T06:30:25.601 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2024-02-21T06:30:29.596 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:29.602 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:29.704 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2024-02-21T06:30:29.710 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2024-02-21T06:30:29.811 INFO:teuthology.orchestra.console.smithi003:Power on completed
2024-02-21T06:30:29.911 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish
2024-02-21T06:30:33.706 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T06:30:33.814 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on
2024-02-21T06:30:33.915 INFO:teuthology.orchestra.console.smithi038:Power on completed
2024-02-21T06:30:34.016 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2024-02-21T06:35:16.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:35:16.283 ERROR:teuthology.orchestra.connection:Error authenticating with smithi038.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-02-21T06:35:20.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:35:23.772 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-02-21T06:35:29.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:35:32.828 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-02-21T06:35:38.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:35:41.884 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-02-21T06:35:47.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:35:50.940 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-02-21T06:35:56.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:35:59.996 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-02-21T06:36:05.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:36:20.286 WARNING:teuthology.provision.fog.smithi038:timed out
2024-02-21T06:36:26.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:36:44.732 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2024-02-21T06:36:50.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:36:52.674 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-21T06:36:52.675 DEBUG:teuthology.orchestra.run.smithi038:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-21T06:36:55.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:36:58.208 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:37:04.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:37:06.000 WARNING:teuthology.provision.fog.smithi003:timed out
2024-02-21T06:37:07.260 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:37:12.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:37:12.234 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-21T06:37:12.235 DEBUG:teuthology.orchestra.run.smithi003:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-21T06:37:12.567 INFO:teuthology.provision.fog.smithi003:Node is ready
2024-02-21T06:37:12.567 DEBUG:teuthology.orchestra.run.smithi003:> hostname
2024-02-21T06:37:12.582 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com
2024-02-21T06:37:12.582 DEBUG:teuthology.orchestra.run.smithi003:> grep smithi003.front.sepia.ceph.com /etc/hosts
2024-02-21T06:37:12.638 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003
2024-02-21T06:37:12.638 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003
2024-02-21T06:37:12.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hosts
2024-02-21T06:37:12.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/172.21.15.3/172.21.15.3/g' /etc/hosts
2024-02-21T06:37:12.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hostname
2024-02-21T06:37:12.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003
2024-02-21T06:37:12.990 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release
2024-02-21T06:37:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream"
2024-02-21T06:37:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9"
2024-02-21T06:37:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T06:37:13.045 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T06:37:13.046 INFO:teuthology.provision.fog.smithi003:Deploy complete!
2024-02-21T06:37:13.047 DEBUG:teuthology.parallel:result is None
2024-02-21T06:37:13.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:37:13.776 INFO:teuthology.provision.fog.smithi038:Node is ready
2024-02-21T06:37:13.776 DEBUG:teuthology.orchestra.run.smithi038:> hostname
2024-02-21T06:37:13.795 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2024-02-21T06:37:13.795 DEBUG:teuthology.orchestra.run.smithi038:> grep smithi038.front.sepia.ceph.com /etc/hosts
2024-02-21T06:37:13.854 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2024-02-21T06:37:13.854 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038
2024-02-21T06:37:13.997 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hosts
2024-02-21T06:37:14.033 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/172.21.15.38/172.21.15.38/g' /etc/hosts
2024-02-21T06:37:14.109 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hostname
2024-02-21T06:37:14.183 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038
2024-02-21T06:37:14.255 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release
2024-02-21T06:37:14.312 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream"
2024-02-21T06:37:14.312 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9"
2024-02-21T06:37:14.312 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos"
2024-02-21T06:37:14.312 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora"
2024-02-21T06:37:14.312 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9"
2024-02-21T06:37:14.312 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9"
2024-02-21T06:37:14.313 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T06:37:14.313 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31"
2024-02-21T06:37:14.313 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon"
2024-02-21T06:37:14.313 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T06:37:14.313 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/"
2024-02-21T06:37:14.313 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T06:37:14.313 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T06:37:14.313 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T06:37:14.313 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2024-02-21T06:37:14.314 DEBUG:teuthology.parallel:result is None
2024-02-21T06:37:16.316 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:37:22.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:37:25.376 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:37:31.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:37:34.428 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:37:40.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:37:43.484 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:37:49.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:37:52.540 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:37:58.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:38:16.892 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:38:22.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:38:25.948 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:38:31.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:38:35.004 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-02-21T06:38:41.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:38:41.237 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-21T06:38:41.237 DEBUG:teuthology.orchestra.run.smithi121:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-21T06:39:01.636 INFO:teuthology.provision.fog.smithi121:Node is ready
2024-02-21T06:39:01.636 DEBUG:teuthology.orchestra.run.smithi121:> hostname
2024-02-21T06:39:01.655 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com
2024-02-21T06:39:01.655 DEBUG:teuthology.orchestra.run.smithi121:> grep smithi121.front.sepia.ceph.com /etc/hosts
2024-02-21T06:39:01.713 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121
2024-02-21T06:39:01.713 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121
2024-02-21T06:39:01.833 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121.front.sepia.ceph.com/smithi121/g' /etc/hosts
2024-02-21T06:39:01.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/172.21.15.121/172.21.15.121/g' /etc/hosts
2024-02-21T06:39:01.932 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121.front.sepia.ceph.com/smithi121/g' /etc/hostname
2024-02-21T06:39:02.000 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121
2024-02-21T06:39:02.067 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release
2024-02-21T06:39:02.122 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream"
2024-02-21T06:39:02.122 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9"
2024-02-21T06:39:02.122 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos"
2024-02-21T06:39:02.122 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora"
2024-02-21T06:39:02.122 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9"
2024-02-21T06:39:02.122 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9"
2024-02-21T06:39:02.122 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T06:39:02.123 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31"
2024-02-21T06:39:02.123 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon"
2024-02-21T06:39:02.123 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T06:39:02.123 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/"
2024-02-21T06:39:02.123 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T06:39:02.123 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T06:39:02.123 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T06:39:02.123 INFO:teuthology.provision.fog.smithi121:Deploy complete!
2024-02-21T06:39:02.124 DEBUG:teuthology.parallel:result is None
2024-02-21T06:39:02.338 INFO:teuthology.lock.ops:Checking smithi003.front.sepia.ceph.com
2024-02-21T06:39:02.339 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-21T06:39:02.370 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com
2024-02-21T06:39:02.371 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-21T06:39:02.386 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com
2024-02-21T06:39:02.387 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-21T06:39:02.404 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: set os type and version on server
2024-02-21T06:39:02.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com'
2024-02-21T06:39:02.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:39:02.505 DEBUG:teuthology.orchestra.run.smithi003:> true
2024-02-21T06:39:02.686 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com'
2024-02-21T06:39:02.686 DEBUG:teuthology.orchestra.run.smithi003:> uname -m
2024-02-21T06:39:02.741 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64
2024-02-21T06:39:02.741 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T06:39:02.796 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T06:39:02.797 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2024-02-21T06:39:02.837 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server
2024-02-21T06:39:02.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2024-02-21T06:39:02.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:39:02.940 DEBUG:teuthology.orchestra.run.smithi038:> true
2024-02-21T06:39:03.124 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com'
2024-02-21T06:39:03.124 DEBUG:teuthology.orchestra.run.smithi038:> uname -m
2024-02-21T06:39:03.179 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64
2024-02-21T06:39:03.180 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release
2024-02-21T06:39:03.234 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream"
2024-02-21T06:39:03.234 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T06:39:03.235 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T06:39:03.236 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2024-02-21T06:39:03.262 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server
2024-02-21T06:39:03.262 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com'
2024-02-21T06:39:03.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T06:39:03.347 DEBUG:teuthology.orchestra.run.smithi121:> true
2024-02-21T06:39:03.415 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com'
2024-02-21T06:39:03.416 DEBUG:teuthology.orchestra.run.smithi121:> uname -m
2024-02-21T06:39:03.470 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64
2024-02-21T06:39:03.471 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon"
2024-02-21T06:39:03.526 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T06:39:03.527 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/"
2024-02-21T06:39:03.527 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T06:39:03.527 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T06:39:03.527 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T06:39:03.527 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2024-02-21T06:39:03.554 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:39:03.636 INFO:teuthology.dispatcher.supervisor:Running job 7569231
2024-02-21T06:39:03.636 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-21_06:28:57-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7569231 --name vshankar-2024-02-21_06:28:57-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsync-tester}} -- /home/teuthworker/archive/vshankar-2024-02-21_06:28:57-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7569231/orig.config.yaml
2024-02-21T06:39:03.638 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-21_06:28:57-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7569231
2024-02-21T06:39:03.639 INFO:teuthology.dispatcher.supervisor:Job PID: 4056202
2024-02-21T06:39:03.639 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-02-21T06:41:03.641 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:43:03.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:45:03.776 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:47:03.844 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:49:03.916 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:51:03.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:53:04.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:55:04.124 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:57:04.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T06:59:04.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:01:04.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:03:04.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:05:04.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:07:04.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:09:04.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:11:04.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:13:04.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:15:04.921 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T07:15:04.986 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1