2024-01-02T20:33:59.713 INFO:root:teuthology version: 0.0.1.dev250+gee17c8b
2024-01-02T20:33:59.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-02T20:33:59.883 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f6973daef40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f6974e13c10>, signals=[15])
2024-01-02T20:33:59.883 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging
2024-01-02T20:33:59.883 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server
2024-01-02T20:33:59.883 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2024-01-02T20:33:59.906 INFO:teuthology.lock.ops:Start node 'smithi022.front.sepia.ceph.com' reimaging
2024-01-02T20:33:59.906 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: reset os type and version on server
2024-01-02T20:33:59.906 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2024-01-02T20:33:59.922 INFO:teuthology.lock.ops:Start node 'smithi093.front.sepia.ceph.com' reimaging
2024-01-02T20:33:59.922 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: reset os type and version on server
2024-01-02T20:33:59.922 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2024-01-02T20:34:00.009 DEBUG:teuthology.provision.fog.smithi002:Requesting image smithi_centos_9.stream (ID 10)
2024-01-02T20:34:00.019 DEBUG:teuthology.provision.fog.smithi022:Requesting image smithi_centos_9.stream (ID 10)
2024-01-02T20:34:00.043 DEBUG:teuthology.provision.fog.smithi093:Requesting image smithi_centos_9.stream (ID 10)
2024-01-02T20:34:00.051 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 9.stream
2024-01-02T20:34:00.057 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of centos 9.stream
2024-01-02T20:34:00.089 INFO:teuthology.provision.fog.smithi093:Scheduling deploy of centos 9.stream
2024-01-02T20:34:00.281 INFO:teuthology.orchestra.console.smithi002:Power off
2024-01-02T20:34:00.282 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-02T20:34:00.292 INFO:teuthology.orchestra.console.smithi022:Power off
2024-01-02T20:34:00.293 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-02T20:34:00.306 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off
2024-01-02T20:34:00.307 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:00.316 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off
2024-01-02T20:34:00.316 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:00.327 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2024-01-02T20:34:00.339 INFO:teuthology.orchestra.console.smithi093:Power off
2024-01-02T20:34:00.339 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-02T20:34:00.349 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-01-02T20:34:00.361 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off
2024-01-02T20:34:00.361 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:00.386 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off
2024-01-02T20:34:00.428 INFO:teuthology.orchestra.console.smithi002:Power off completed
2024-01-02T20:34:00.449 INFO:teuthology.orchestra.console.smithi022:Power off completed
2024-01-02T20:34:00.486 INFO:teuthology.orchestra.console.smithi093:Power off completed
2024-01-02T20:34:00.529 INFO:teuthology.orchestra.console.smithi002:Power on
2024-01-02T20:34:00.531 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-02T20:34:00.550 INFO:teuthology.orchestra.console.smithi022:Power on
2024-01-02T20:34:00.550 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-02T20:34:00.556 DEBUG:teuthology.orchestra.console.smithi002:power on output: Chassis Power Control: Up/On
2024-01-02T20:34:00.556 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:00.570 DEBUG:teuthology.orchestra.console.smithi022:power on output: Chassis Power Control: Up/On
2024-01-02T20:34:00.570 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:00.578 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2024-01-02T20:34:00.587 INFO:teuthology.orchestra.console.smithi093:Power on
2024-01-02T20:34:00.587 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-02T20:34:00.595 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-01-02T20:34:00.607 DEBUG:teuthology.orchestra.console.smithi093:power on output: Chassis Power Control: Up/On
2024-01-02T20:34:00.607 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:00.626 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off
2024-01-02T20:34:04.581 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:04.596 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:04.627 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:04.693 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on
2024-01-02T20:34:04.707 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-01-02T20:34:04.738 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on
2024-01-02T20:34:04.794 INFO:teuthology.orchestra.console.smithi002:Power on completed
2024-01-02T20:34:04.839 INFO:teuthology.orchestra.console.smithi093:Power on completed
2024-01-02T20:34:04.896 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish
2024-01-02T20:34:04.940 INFO:teuthology.provision.fog.smithi093:Waiting for deploy to finish
2024-01-02T20:34:08.712 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:34:08.823 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on
2024-01-02T20:34:08.923 INFO:teuthology.orchestra.console.smithi022:Power on completed
2024-01-02T20:34:09.024 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish
2024-01-02T20:36:35.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:36:35.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:36:39.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:36:54.204 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-01-02T20:36:58.300 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-01-02T20:37:00.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:37:03.260 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-01-02T20:37:04.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:37:07.356 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-01-02T20:37:09.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:37:12.316 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-01-02T20:37:13.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:37:16.412 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-01-02T20:37:18.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:37:21.372 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-01-02T20:37:22.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:37:27.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:37:35.887 ERROR:teuthology.orchestra.connection:Error authenticating with smithi093.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_ee17c8bb72e66724e53b1ff6ca81f0cad937bff6/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ee17c8bb72e66724e53b1ff6ca81f0cad937bff6/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 457, in connect
    server_key = t.get_remote_server_key()
  File "/home/teuthworker/src/git.ceph.com_teuthology_ee17c8bb72e66724e53b1ff6ca81f0cad937bff6/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 918, in get_remote_server_key
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2024-01-02T20:38:18.941 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2024-01-02T20:38:22.420 WARNING:teuthology.provision.fog.smithi022:timed out
2024-01-02T20:38:27.382 WARNING:teuthology.provision.fog.smithi002:timed out
2024-01-02T20:38:28.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:38:28.676 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-02T20:38:28.677 DEBUG:teuthology.orchestra.run.smithi022:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-02T20:38:29.000 INFO:teuthology.provision.fog.smithi022:Node is ready
2024-01-02T20:38:29.000 DEBUG:teuthology.orchestra.run.smithi022:> hostname
2024-01-02T20:38:29.016 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com
2024-01-02T20:38:29.016 DEBUG:teuthology.orchestra.run.smithi022:> grep smithi022.front.sepia.ceph.com /etc/hosts
2024-01-02T20:38:29.072 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2024-01-02T20:38:29.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022
2024-01-02T20:38:29.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hosts
2024-01-02T20:38:29.207 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/172.21.15.22/172.21.15.22/g' /etc/hosts
2024-01-02T20:38:29.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hostname
2024-01-02T20:38:29.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022
2024-01-02T20:38:29.419 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release
2024-01-02T20:38:29.475 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream"
2024-01-02T20:38:29.475 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9"
2024-01-02T20:38:29.475 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos"
2024-01-02T20:38:29.475 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora"
2024-01-02T20:38:29.475 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9"
2024-01-02T20:38:29.475 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9"
2024-01-02T20:38:29.475 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-02T20:38:29.476 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31"
2024-01-02T20:38:29.476 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon"
2024-01-02T20:38:29.476 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-02T20:38:29.476 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/"
2024-01-02T20:38:29.476 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-02T20:38:29.476 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-02T20:38:29.476 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-02T20:38:29.477 INFO:teuthology.provision.fog.smithi022:Deploy complete!
2024-01-02T20:38:29.480 DEBUG:teuthology.parallel:result is None
2024-01-02T20:38:33.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:38:33.640 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-02T20:38:33.640 DEBUG:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-02T20:38:33.977 INFO:teuthology.provision.fog.smithi002:Node is ready
2024-01-02T20:38:33.977 DEBUG:teuthology.orchestra.run.smithi002:> hostname
2024-01-02T20:38:33.993 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com
2024-01-02T20:38:33.993 DEBUG:teuthology.orchestra.run.smithi002:> grep smithi002.front.sepia.ceph.com /etc/hosts
2024-01-02T20:38:34.050 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002
2024-01-02T20:38:34.051 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002
2024-01-02T20:38:34.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hosts
2024-01-02T20:38:34.194 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/172.21.15.2/172.21.15.2/g' /etc/hosts
2024-01-02T20:38:34.267 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hostname
2024-01-02T20:38:34.340 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002
2024-01-02T20:38:34.412 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release
2024-01-02T20:38:34.468 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream"
2024-01-02T20:38:34.468 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9"
2024-01-02T20:38:34.468 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos"
2024-01-02T20:38:34.468 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora"
2024-01-02T20:38:34.468 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9"
2024-01-02T20:38:34.468 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9"
2024-01-02T20:38:34.468 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-02T20:38:34.469 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31"
2024-01-02T20:38:34.469 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon"
2024-01-02T20:38:34.469 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-02T20:38:34.469 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/"
2024-01-02T20:38:34.469 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-02T20:38:34.469 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-02T20:38:34.469 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-02T20:38:34.470 INFO:teuthology.provision.fog.smithi002:Deploy complete!
2024-01-02T20:38:34.470 DEBUG:teuthology.parallel:result is None
2024-01-02T20:38:39.898 WARNING:teuthology.provision.fog.smithi093:timed out
2024-01-02T20:38:45.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:38:46.144 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-02T20:38:46.144 DEBUG:teuthology.orchestra.run.smithi093:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-02T20:38:46.463 INFO:teuthology.provision.fog.smithi093:Node is ready
2024-01-02T20:38:46.463 DEBUG:teuthology.orchestra.run.smithi093:> hostname
2024-01-02T20:38:46.478 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093.front.sepia.ceph.com
2024-01-02T20:38:46.478 DEBUG:teuthology.orchestra.run.smithi093:> grep smithi093.front.sepia.ceph.com /etc/hosts
2024-01-02T20:38:46.535 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93 smithi093.front.sepia.ceph.com smithi093
2024-01-02T20:38:46.535 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname smithi093
2024-01-02T20:38:46.648 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/smithi093.front.sepia.ceph.com/smithi093/g' /etc/hosts
2024-01-02T20:38:46.676 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/172.21.15.93/172.21.15.93/g' /etc/hosts
2024-01-02T20:38:46.745 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/smithi093.front.sepia.ceph.com/smithi093/g' /etc/hostname
2024-01-02T20:38:46.813 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname smithi093
2024-01-02T20:38:46.880 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release
2024-01-02T20:38:46.935 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="CentOS Stream"
2024-01-02T20:38:46.935 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="9"
2024-01-02T20:38:46.935 INFO:teuthology.orchestra.run.smithi093.stdout:ID="centos"
2024-01-02T20:38:46.935 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="rhel fedora"
2024-01-02T20:38:46.935 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="9"
2024-01-02T20:38:46.935 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el9"
2024-01-02T20:38:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-02T20:38:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31"
2024-01-02T20:38:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:LOGO="fedora-logo-icon"
2024-01-02T20:38:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-02T20:38:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://centos.org/"
2024-01-02T20:38:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-02T20:38:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-02T20:38:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-02T20:38:46.936 INFO:teuthology.provision.fog.smithi093:Deploy complete!
2024-01-02T20:38:46.937 DEBUG:teuthology.parallel:result is None
2024-01-02T20:38:47.164 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com
2024-01-02T20:38:47.164 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-02T20:38:47.282 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com
2024-01-02T20:38:47.282 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-02T20:38:47.306 INFO:teuthology.lock.ops:Checking smithi093.front.sepia.ceph.com
2024-01-02T20:38:47.307 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-02T20:38:47.324 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server
2024-01-02T20:38:47.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com'
2024-01-02T20:38:47.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:38:47.424 DEBUG:teuthology.orchestra.run.smithi002:> true
2024-01-02T20:38:47.606 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com'
2024-01-02T20:38:47.606 DEBUG:teuthology.orchestra.run.smithi002:> uname -m
2024-01-02T20:38:47.662 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64
2024-01-02T20:38:47.662 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release
2024-01-02T20:38:47.717 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream"
2024-01-02T20:38:47.717 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9"
2024-01-02T20:38:47.717 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos"
2024-01-02T20:38:47.717 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora"
2024-01-02T20:38:47.717 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-02T20:38:47.718 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-02T20:38:47.718 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2024-01-02T20:38:47.740 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server
2024-01-02T20:38:47.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com'
2024-01-02T20:38:47.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:38:47.840 DEBUG:teuthology.orchestra.run.smithi022:> true
2024-01-02T20:38:48.022 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com'
2024-01-02T20:38:48.022 DEBUG:teuthology.orchestra.run.smithi022:> uname -m
2024-01-02T20:38:48.077 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64
2024-01-02T20:38:48.077 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release
2024-01-02T20:38:48.132 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream"
2024-01-02T20:38:48.132 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9"
2024-01-02T20:38:48.132 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-02T20:38:48.133 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-02T20:38:48.134 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2024-01-02T20:38:48.165 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: set os type and version on server
2024-01-02T20:38:48.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com'
2024-01-02T20:38:48.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-02T20:38:48.252 DEBUG:teuthology.orchestra.run.smithi093:> true
2024-01-02T20:38:48.320 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com'
2024-01-02T20:38:48.320 DEBUG:teuthology.orchestra.run.smithi093:> uname -m
2024-01-02T20:38:48.376 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64
2024-01-02T20:38:48.376 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release
2024-01-02T20:38:48.432 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="CentOS Stream"
2024-01-02T20:38:48.432 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="9"
2024-01-02T20:38:48.432 INFO:teuthology.orchestra.run.smithi093.stdout:ID="centos"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="rhel fedora"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="9"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el9"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:LOGO="fedora-logo-icon"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://centos.org/"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-02T20:38:48.433 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-02T20:38:48.434 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2024-01-02T20:38:48.448 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-02T20:38:48.543 INFO:teuthology.dispatcher.supervisor:Running job 7503707
2024-01-02T20:38:48.543 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_ee17c8bb72e66724e53b1ff6ca81f0cad937bff6/virtualenv/bin/teuthology -v --owner zmc --archive /home/teuthworker/archive/zack-2024-01-02_20:18:26-smoke-main-distro-default-smithi/7503707 --name zack-2024-01-02_20:18:26-smoke-main-distro-default-smithi --description smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} tasks/{0-install test/kclient_workunit_direct_io}} -- /home/teuthworker/archive/zack-2024-01-02_20:18:26-smoke-main-distro-default-smithi/7503707/orig.config.yaml
2024-01-02T20:38:48.546 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/zack-2024-01-02_20:18:26-smoke-main-distro-default-smithi/7503707
2024-01-02T20:38:48.547 INFO:teuthology.dispatcher.supervisor:Job PID: 3567361
2024-01-02T20:38:48.547 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-02T20:40:48.649 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-02T20:42:48.740 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-02T20:42:48.814 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-02T20:42:48.931 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-02T20:42:48.932 INFO:teuthology.nuke:Checking targets against current locks
2024-01-02T20:42:48.983 DEBUG:teuthology.nuke:shortname: smithi002
2024-01-02T20:42:48.983 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-02T20:42:49.005 DEBUG:teuthology.nuke:shortname: smithi022
2024-01-02T20:42:49.005 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-02T20:42:49.009 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/zack-2024-01-02_20:18:26-smoke-main-distro-default-smithi/7503707', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-02 20:33:58.646237', 'locked_by': 'zmc', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCeYujQvCE7AoUZAxxd7RUvyLmNgWW2RX1c+wqfqHESyiQPkfZvkhCR2REH9XJcaj8Bs6PNrYWhoQYfXTnumfdM='}
2024-01-02T20:42:49.020 DEBUG:teuthology.nuke:shortname: smithi093
2024-01-02T20:42:49.020 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-02T20:42:49.033 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/zack-2024-01-02_20:18:26-smoke-main-distro-default-smithi/7503707', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-02 20:33:58.647196', 'locked_by': 'zmc', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBuyGgS+r22c+DKHUV3AOo/W49bYoaPUo18uGzahOB7UxRyLzOG1ilUGQHiaIwbGoyPXsANjbN/tlleCNuxzICk='}
2024-01-02T20:42:49.041 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/zack-2024-01-02_20:18:26-smoke-main-distro-default-smithi/7503707', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-02 20:33:58.648128', 'locked_by': 'zmc', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCGlbthE78gSa6sPD380KZXMbRs26CkJb5gR6TxBGxUqT9RsmbXsK9YriQdpsKiH0prxnM1gag4g3xHLgPeEOX8='}
2024-01-02T20:42:49.065 INFO:teuthology.orchestra.console.smithi002:Power off
2024-01-02T20:42:49.066 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-02T20:42:49.089 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off
2024-01-02T20:42:49.089 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:42:49.100 INFO:teuthology.orchestra.console.smithi022:Power off
2024-01-02T20:42:49.100 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-02T20:42:49.111 INFO:teuthology.orchestra.console.smithi093:Power off
2024-01-02T20:42:49.111 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-02T20:42:49.118 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2024-01-02T20:42:49.119 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off
2024-01-02T20:42:49.119 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:42:49.131 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off
2024-01-02T20:42:49.131 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-02T20:42:49.139 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-01-02T20:42:49.151 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off
2024-01-02T20:42:49.219 INFO:teuthology.orchestra.console.smithi002:Power off completed
2024-01-02T20:42:49.240 INFO:teuthology.orchestra.console.smithi022:Power off completed
2024-01-02T20:42:49.252 INFO:teuthology.orchestra.console.smithi093:Power off completed
2024-01-02T20:42:49.372 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com
2024-01-02T20:42:49.396 INFO:teuthology.lock.ops:unlocked: smithi022.front.sepia.ceph.com
2024-01-02T20:42:49.405 INFO:teuthology.lock.ops:unlocked: smithi093.front.sepia.ceph.com