2024-04-14T22:09:09.982 INFO:root:teuthology version: 0.0.1.dev267+g6c63784
2024-04-14T22:09:09.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:09:10.146 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fbaf6a94f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fbaf7af5c10>, signals=[15])
2024-04-14T22:09:10.146 INFO:teuthology.lock.ops:Start node 'smithi110.front.sepia.ceph.com' reimaging
2024-04-14T22:09:10.146 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: reset os type and version on server
2024-04-14T22:09:10.146 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2024-04-14T22:09:10.180 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging
2024-04-14T22:09:10.181 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server
2024-04-14T22:09:10.181 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2024-04-14T22:09:10.205 INFO:teuthology.lock.ops:Start node 'smithi195.front.sepia.ceph.com' reimaging
2024-04-14T22:09:10.205 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: reset os type and version on server
2024-04-14T22:09:10.205 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server
2024-04-14T22:09:10.283 DEBUG:teuthology.provision.fog.smithi110:Requesting image smithi_centos_9.stream (ID 10)
2024-04-14T22:09:10.320 DEBUG:teuthology.provision.fog.smithi167:Requesting image smithi_centos_9.stream (ID 10)
2024-04-14T22:09:10.326 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 9.stream
2024-04-14T22:09:10.338 DEBUG:teuthology.provision.fog.smithi195:Requesting image smithi_centos_9.stream (ID 10)
2024-04-14T22:09:10.378 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 9.stream
2024-04-14T22:09:10.383 INFO:teuthology.provision.fog.smithi195:Scheduling deploy of centos 9.stream
2024-04-14T22:09:10.588 INFO:teuthology.orchestra.console.smithi110:Power off
2024-04-14T22:09:10.588 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-14T22:09:10.614 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off
2024-04-14T22:09:10.614 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:10.628 INFO:teuthology.orchestra.console.smithi167:Power off
2024-04-14T22:09:10.628 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-14T22:09:10.642 INFO:teuthology.orchestra.console.smithi195:Power off
2024-04-14T22:09:10.642 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-14T22:09:10.649 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2024-04-14T22:09:10.650 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off
2024-04-14T22:09:10.650 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:10.664 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off
2024-04-14T22:09:10.665 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:10.673 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2024-04-14T22:09:10.687 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2024-04-14T22:09:14.654 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:14.674 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:14.688 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:14.765 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2024-04-14T22:09:14.783 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2024-04-14T22:09:14.797 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2024-04-14T22:09:18.770 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:18.784 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:18.797 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:18.878 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2024-04-14T22:09:18.893 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2024-04-14T22:09:18.907 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2024-04-14T22:09:18.979 INFO:teuthology.orchestra.console.smithi110:Power off completed
2024-04-14T22:09:19.007 INFO:teuthology.orchestra.console.smithi195:Power off completed
2024-04-14T22:09:19.080 INFO:teuthology.orchestra.console.smithi110:Power on
2024-04-14T22:09:19.080 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-14T22:09:19.102 DEBUG:teuthology.orchestra.console.smithi110:power on output: Chassis Power Control: Up/On
2024-04-14T22:09:19.102 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:19.110 INFO:teuthology.orchestra.console.smithi195:Power on
2024-04-14T22:09:19.111 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-14T22:09:19.128 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2024-04-14T22:09:19.135 DEBUG:teuthology.orchestra.console.smithi195:power on output: Chassis Power Control: Up/On
2024-04-14T22:09:19.135 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:19.155 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2024-04-14T22:09:22.898 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:23.006 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2024-04-14T22:09:23.107 INFO:teuthology.orchestra.console.smithi167:Power off completed
2024-04-14T22:09:23.129 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:23.156 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:23.208 INFO:teuthology.orchestra.console.smithi167:Power on
2024-04-14T22:09:23.209 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-14T22:09:23.231 DEBUG:teuthology.orchestra.console.smithi167:power on output: Chassis Power Control: Up/On
2024-04-14T22:09:23.231 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:23.240 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2024-04-14T22:09:23.254 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2024-04-14T22:09:23.264 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2024-04-14T22:09:27.242 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:27.254 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:27.265 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-14T22:09:27.353 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2024-04-14T22:09:27.363 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2024-04-14T22:09:27.373 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2024-04-14T22:09:27.454 INFO:teuthology.orchestra.console.smithi110:Power on completed
2024-04-14T22:09:27.463 INFO:teuthology.orchestra.console.smithi167:Power on completed
2024-04-14T22:09:27.474 INFO:teuthology.orchestra.console.smithi195:Power on completed
2024-04-14T22:09:27.555 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish
2024-04-14T22:09:27.564 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish
2024-04-14T22:09:27.575 INFO:teuthology.provision.fog.smithi195:Waiting for deploy to finish
2024-04-14T22:11:58.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:11:58.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:11:58.607 ERROR:teuthology.orchestra.connection:Error authenticating with smithi195.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-04-14T22:12:13.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:12:16.638 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-14T22:12:22.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:12:25.694 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-14T22:12:31.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:12:34.750 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-14T22:12:40.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:12:43.806 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-14T22:12:49.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:12:52.866 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-14T22:12:58.576 ERROR:teuthology.orchestra.connection:Error authenticating with smithi110.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 457, in connect
    server_key = t.get_remote_server_key()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 918, in get_remote_server_key
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2024-04-14T22:12:58.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:13:01.918 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-14T22:13:02.610 WARNING:teuthology.provision.fog.smithi195:timed out
2024-04-14T22:13:07.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:13:08.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:13:10.978 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2024-04-14T22:13:16.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:13:26.946 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2024-04-14T22:13:32.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:13:36.030 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2024-04-14T22:13:42.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:13:42.362 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-14T22:13:42.363 DEBUG:teuthology.orchestra.run.smithi195:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-14T22:13:46.402 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2024-04-14T22:14:02.582 WARNING:teuthology.provision.fog.smithi110:timed out
2024-04-14T22:14:02.691 INFO:teuthology.provision.fog.smithi195:Node is ready
2024-04-14T22:14:02.691 DEBUG:teuthology.orchestra.run.smithi195:> hostname
2024-04-14T22:14:02.710 INFO:teuthology.orchestra.run.smithi195.stdout:smithi195.front.sepia.ceph.com
2024-04-14T22:14:02.710 DEBUG:teuthology.orchestra.run.smithi195:> grep smithi195.front.sepia.ceph.com /etc/hosts
2024-04-14T22:14:02.768 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.15.195 smithi195.front.sepia.ceph.com smithi195
2024-04-14T22:14:02.768 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195
2024-04-14T22:14:02.885 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hosts
2024-04-14T22:14:02.914 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/172.21.15.195/172.21.15.195/g' /etc/hosts
2024-04-14T22:14:02.983 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hostname
2024-04-14T22:14:03.051 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195
2024-04-14T22:14:03.118 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release
2024-04-14T22:14:03.174 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-14T22:14:03.175 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/"
2024-04-14T22:14:03.176 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-14T22:14:03.176 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-14T22:14:03.176 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-14T22:14:03.177 INFO:teuthology.provision.fog.smithi195:Deploy complete!
2024-04-14T22:14:03.178 DEBUG:teuthology.parallel:result is None
2024-04-14T22:14:08.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:14:08.834 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-14T22:14:08.834 DEBUG:teuthology.orchestra.run.smithi110:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-14T22:14:09.167 INFO:teuthology.provision.fog.smithi110:Node is ready
2024-04-14T22:14:09.167 DEBUG:teuthology.orchestra.run.smithi110:> hostname
2024-04-14T22:14:09.182 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com
2024-04-14T22:14:09.182 DEBUG:teuthology.orchestra.run.smithi110:> grep smithi110.front.sepia.ceph.com /etc/hosts
2024-04-14T22:14:09.238 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110
2024-04-14T22:14:09.238 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110
2024-04-14T22:14:09.353 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hosts
2024-04-14T22:14:09.382 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/172.21.15.110/172.21.15.110/g' /etc/hosts
2024-04-14T22:14:09.449 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hostname
2024-04-14T22:14:09.517 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110
2024-04-14T22:14:09.583 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release
2024-04-14T22:14:09.638 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream"
2024-04-14T22:14:09.638 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9"
2024-04-14T22:14:09.638 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos"
2024-04-14T22:14:09.638 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora"
2024-04-14T22:14:09.638 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9"
2024-04-14T22:14:09.638 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9"
2024-04-14T22:14:09.638 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-14T22:14:09.638 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2024-04-14T22:14:09.639 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon"
2024-04-14T22:14:09.639 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-14T22:14:09.639 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/"
2024-04-14T22:14:09.639 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-14T22:14:09.639 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-14T22:14:09.639 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-14T22:14:09.639 INFO:teuthology.provision.fog.smithi110:Deploy complete!
2024-04-14T22:14:09.640 DEBUG:teuthology.parallel:result is None
2024-04-14T22:14:16.982 WARNING:teuthology.provision.fog.smithi167:timed out
2024-04-14T22:14:22.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:14:23.472 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-14T22:14:23.472 DEBUG:teuthology.orchestra.run.smithi167:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-14T22:14:23.828 INFO:teuthology.provision.fog.smithi167:Node is ready
2024-04-14T22:14:23.828 DEBUG:teuthology.orchestra.run.smithi167:> hostname
2024-04-14T22:14:23.843 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com
2024-04-14T22:14:23.843 DEBUG:teuthology.orchestra.run.smithi167:> grep smithi167.front.sepia.ceph.com /etc/hosts
2024-04-14T22:14:23.900 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167
2024-04-14T22:14:23.901 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167
2024-04-14T22:14:24.022 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hosts
2024-04-14T22:14:24.051 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/172.21.15.167/172.21.15.167/g' /etc/hosts
2024-04-14T22:14:24.119 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hostname
2024-04-14T22:14:24.187 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167
2024-04-14T22:14:24.253 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release
2024-04-14T22:14:24.308 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream"
2024-04-14T22:14:24.308 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9"
2024-04-14T22:14:24.308 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos"
2024-04-14T22:14:24.308 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora"
2024-04-14T22:14:24.308 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-14T22:14:24.309 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-14T22:14:24.309 INFO:teuthology.provision.fog.smithi167:Deploy complete!
2024-04-14T22:14:24.310 DEBUG:teuthology.parallel:result is None
2024-04-14T22:14:24.532 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com
2024-04-14T22:14:24.532 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-14T22:14:24.559 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com
2024-04-14T22:14:24.559 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-14T22:14:24.583 INFO:teuthology.lock.ops:Checking smithi195.front.sepia.ceph.com
2024-04-14T22:14:24.583 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-14T22:14:24.601 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server
2024-04-14T22:14:24.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com'
2024-04-14T22:14:24.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:14:24.697 DEBUG:teuthology.orchestra.run.smithi110:> true
2024-04-14T22:14:24.864 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com'
2024-04-14T22:14:24.865 DEBUG:teuthology.orchestra.run.smithi110:> uname -m
2024-04-14T22:14:24.919 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64
2024-04-14T22:14:24.920 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release
2024-04-14T22:14:24.974 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream"
2024-04-14T22:14:24.974 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9"
2024-04-14T22:14:24.974 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos"
2024-04-14T22:14:24.974 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora"
2024-04-14T22:14:24.974 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9"
2024-04-14T22:14:24.974 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9"
2024-04-14T22:14:24.974 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-14T22:14:24.975 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2024-04-14T22:14:24.975 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon"
2024-04-14T22:14:24.975 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-14T22:14:24.975 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/"
2024-04-14T22:14:24.975 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-14T22:14:24.975 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-14T22:14:24.975 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-14T22:14:24.975 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2024-04-14T22:14:24.992 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server
2024-04-14T22:14:24.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2024-04-14T22:14:24.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:14:25.094 DEBUG:teuthology.orchestra.run.smithi167:> true
2024-04-14T22:14:25.162 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com'
2024-04-14T22:14:25.163 DEBUG:teuthology.orchestra.run.smithi167:> uname -m
2024-04-14T22:14:25.218 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64
2024-04-14T22:14:25.218 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release
2024-04-14T22:14:25.275 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream"
2024-04-14T22:14:25.275 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9"
2024-04-14T22:14:25.275 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos"
2024-04-14T22:14:25.275 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora"
2024-04-14T22:14:25.275 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9"
2024-04-14T22:14:25.275 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9"
2024-04-14T22:14:25.276 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-14T22:14:25.276 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2024-04-14T22:14:25.276 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon"
2024-04-14T22:14:25.276 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-14T22:14:25.277 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/"
2024-04-14T22:14:25.277 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-14T22:14:25.277 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-14T22:14:25.277 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-14T22:14:25.278 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2024-04-14T22:14:25.300 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: set os type and version on server
2024-04-14T22:14:25.301 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com'
2024-04-14T22:14:25.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-14T22:14:25.391 DEBUG:teuthology.orchestra.run.smithi195:> true
2024-04-14T22:14:25.463 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com'
2024-04-14T22:14:25.463 DEBUG:teuthology.orchestra.run.smithi195:> uname -m
2024-04-14T22:14:25.521 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64
2024-04-14T22:14:25.522 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release
2024-04-14T22:14:25.577 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/"
2024-04-14T22:14:25.578 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-14T22:14:25.579 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-14T22:14:25.579 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-14T22:14:25.579 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server
2024-04-14T22:14:25.600 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:14:25.707 INFO:teuthology.dispatcher.supervisor:Running job 7656481
2024-04-14T22:14:25.708 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-14_21:24:03-fs-squid-distro-default-smithi/7656481 --name teuthology-2024-04-14_21:24:03-fs-squid-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} -- /home/teuthworker/archive/teuthology-2024-04-14_21:24:03-fs-squid-distro-default-smithi/7656481/orig.config.yaml
2024-04-14T22:14:25.710 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-14_21:24:03-fs-squid-distro-default-smithi/7656481
2024-04-14T22:14:25.711 INFO:teuthology.dispatcher.supervisor:Job PID: 2183908
2024-04-14T22:14:25.711 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-04-14T22:16:25.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:18:25.787 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:20:25.867 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:22:25.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:24:26.010 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:26:26.081 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:28:26.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:30:26.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:32:26.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-14T22:32:26.385 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-04-14T22:32:26.571 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi110 but it was locked by another job: /home/teuthworker/archive/teuthology-2024-04-14_21:24:03-fs-squid-distro-default-smithi/7656512
2024-04-14T22:32:26.571 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi167 but it was locked by another job: /home/teuthworker/archive/teuthology-2024-04-14_21:24:03-fs-squid-distro-default-smithi/7656505