2024-04-17T14:39:57.975 INFO:root:teuthology version: 0.0.1.dev267+g6c63784
2024-04-17T14:39:57.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-17T14:39:58.173 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f85aa528f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f85ab55fc10>, signals=[15])
2024-04-17T14:39:58.174 INFO:teuthology.lock.ops:Start node 'smithi088.front.sepia.ceph.com' reimaging
2024-04-17T14:39:58.174 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: reset os type and version on server
2024-04-17T14:39:58.174 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server
2024-04-17T14:39:58.197 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging
2024-04-17T14:39:58.197 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server
2024-04-17T14:39:58.198 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server
2024-04-17T14:39:58.214 INFO:teuthology.lock.ops:Start node 'smithi161.front.sepia.ceph.com' reimaging
2024-04-17T14:39:58.214 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: reset os type and version on server
2024-04-17T14:39:58.214 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2024-04-17T14:39:58.309 DEBUG:teuthology.provision.fog.smithi088:Requesting image smithi_centos_9.stream (ID 10)
2024-04-17T14:39:58.326 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_centos_9.stream (ID 10)
2024-04-17T14:39:58.336 DEBUG:teuthology.provision.fog.smithi161:Requesting image smithi_centos_9.stream (ID 10)
2024-04-17T14:39:58.352 INFO:teuthology.provision.fog.smithi088:Scheduling deploy of centos 9.stream
2024-04-17T14:39:58.370 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 9.stream
2024-04-17T14:39:58.373 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of centos 9.stream
2024-04-17T14:39:59.048 INFO:teuthology.orchestra.console.smithi088:Power off
2024-04-17T14:39:59.048 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-17T14:39:59.070 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off
2024-04-17T14:39:59.070 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:39:59.083 INFO:teuthology.orchestra.console.smithi161:Power off
2024-04-17T14:39:59.083 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-17T14:39:59.095 INFO:teuthology.orchestra.console.smithi112:Power off
2024-04-17T14:39:59.095 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-17T14:39:59.102 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off
2024-04-17T14:39:59.103 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off
2024-04-17T14:39:59.103 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:39:59.118 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off
2024-04-17T14:39:59.119 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:39:59.126 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-04-17T14:39:59.140 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off
2024-04-17T14:39:59.203 INFO:teuthology.orchestra.console.smithi088:Power off completed
2024-04-17T14:39:59.228 INFO:teuthology.orchestra.console.smithi161:Power off completed
2024-04-17T14:39:59.241 INFO:teuthology.orchestra.console.smithi112:Power off completed
2024-04-17T14:39:59.305 INFO:teuthology.orchestra.console.smithi088:Power on
2024-04-17T14:39:59.305 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-17T14:39:59.328 DEBUG:teuthology.orchestra.console.smithi088:power on output: Chassis Power Control: Up/On
2024-04-17T14:39:59.328 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:39:59.339 INFO:teuthology.orchestra.console.smithi161:Power on
2024-04-17T14:39:59.339 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-17T14:39:59.345 INFO:teuthology.orchestra.console.smithi112:Power on
2024-04-17T14:39:59.346 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-17T14:39:59.353 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off
2024-04-17T14:39:59.359 DEBUG:teuthology.orchestra.console.smithi161:power on output: Chassis Power Control: Up/On
2024-04-17T14:39:59.359 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:39:59.366 DEBUG:teuthology.orchestra.console.smithi112:power on output: Chassis Power Control: Up/On
2024-04-17T14:39:59.366 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:39:59.379 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-04-17T14:39:59.385 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off
2024-04-17T14:40:03.358 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:40:03.380 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:40:03.386 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:40:03.467 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off
2024-04-17T14:40:03.490 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-04-17T14:40:03.494 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off
2024-04-17T14:40:07.472 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:40:07.491 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:40:07.498 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-17T14:40:07.580 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on
2024-04-17T14:40:07.599 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-04-17T14:40:07.605 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on
2024-04-17T14:40:07.681 INFO:teuthology.orchestra.console.smithi088:Power on completed
2024-04-17T14:40:07.700 INFO:teuthology.orchestra.console.smithi161:Power on completed
2024-04-17T14:40:07.706 INFO:teuthology.orchestra.console.smithi112:Power on completed
2024-04-17T14:40:07.783 INFO:teuthology.provision.fog.smithi088:Waiting for deploy to finish
2024-04-17T14:40:07.801 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish
2024-04-17T14:40:07.810 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish
2024-04-17T14:44:27.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:44:30.430 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112
2024-04-17T14:44:36.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:44:39.486 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112
2024-04-17T14:44:42.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:44:45.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:44:48.542 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112
2024-04-17T14:44:54.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:44:57.598 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112
2024-04-17T14:45:03.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:45:06.654 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112
2024-04-17T14:45:12.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:45:42.558 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 457, in connect
    server_key = t.get_remote_server_key()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 918, in get_remote_server_key
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2024-04-17T14:46:12.659 WARNING:teuthology.provision.fog.smithi112:timed out
2024-04-17T14:46:13.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:46:16.542 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:46:18.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:46:18.903 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-17T14:46:18.903 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-17T14:46:19.220 INFO:teuthology.provision.fog.smithi112:Node is ready
2024-04-17T14:46:19.220 DEBUG:teuthology.orchestra.run.smithi112:> hostname
2024-04-17T14:46:19.235 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com
2024-04-17T14:46:19.235 DEBUG:teuthology.orchestra.run.smithi112:> grep smithi112.front.sepia.ceph.com /etc/hosts
2024-04-17T14:46:19.290 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112
2024-04-17T14:46:19.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112
2024-04-17T14:46:19.397 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hosts
2024-04-17T14:46:19.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/172.21.15.112/172.21.15.112/g' /etc/hosts
2024-04-17T14:46:19.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hostname
2024-04-17T14:46:19.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112
2024-04-17T14:46:19.625 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release
2024-04-17T14:46:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream"
2024-04-17T14:46:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-17T14:46:19.680 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-17T14:46:19.681 INFO:teuthology.provision.fog.smithi112:Deploy complete!
2024-04-17T14:46:19.683 DEBUG:teuthology.parallel:result is None
2024-04-17T14:46:22.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:46:25.598 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:46:31.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:46:34.654 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:46:40.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:46:43.710 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:46:46.564 WARNING:teuthology.provision.fog.smithi161:timed out
2024-04-17T14:46:49.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:46:52.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:46:52.766 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:46:52.876 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-17T14:46:52.876 DEBUG:teuthology.orchestra.run.smithi161:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-17T14:46:53.199 INFO:teuthology.provision.fog.smithi161:Node is ready
2024-04-17T14:46:53.200 DEBUG:teuthology.orchestra.run.smithi161:> hostname
2024-04-17T14:46:53.215 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com
2024-04-17T14:46:53.233 DEBUG:teuthology.orchestra.run.smithi161:> grep smithi161.front.sepia.ceph.com /etc/hosts
2024-04-17T14:46:53.271 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161
2024-04-17T14:46:53.271 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161
2024-04-17T14:46:53.379 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hosts
2024-04-17T14:46:53.408 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/172.21.15.161/172.21.15.161/g' /etc/hosts
2024-04-17T14:46:53.476 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hostname
2024-04-17T14:46:53.543 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161
2024-04-17T14:46:53.610 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/"
2024-04-17T14:46:53.665 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-17T14:46:53.666 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-17T14:46:53.666 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-17T14:46:53.666 INFO:teuthology.provision.fog.smithi161:Deploy complete!
2024-04-17T14:46:53.666 DEBUG:teuthology.parallel:result is None
2024-04-17T14:46:55.007 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2024-04-17T14:46:58.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:47:01.822 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:47:07.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:47:10.910 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:47:16.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:47:35.454 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:47:41.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:47:44.510 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:47:50.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:47:53.602 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-04-17T14:47:59.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:47:59.864 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-17T14:47:59.864 DEBUG:teuthology.orchestra.run.smithi088:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-17T14:48:15.296 INFO:teuthology.provision.fog.smithi088:Node is ready
2024-04-17T14:48:15.296 DEBUG:teuthology.orchestra.run.smithi088:> hostname
2024-04-17T14:48:15.355 INFO:teuthology.orchestra.run.smithi088.stdout:smithi088.front.sepia.ceph.com
2024-04-17T14:48:15.355 DEBUG:teuthology.orchestra.run.smithi088:> grep smithi088.front.sepia.ceph.com /etc/hosts
2024-04-17T14:48:15.411 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88 smithi088.front.sepia.ceph.com smithi088
2024-04-17T14:48:15.412 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088
2024-04-17T14:48:15.523 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088.front.sepia.ceph.com/smithi088/g' /etc/hosts
2024-04-17T14:48:15.551 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/172.21.15.88/172.21.15.88/g' /etc/hosts
2024-04-17T14:48:15.620 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088.front.sepia.ceph.com/smithi088/g' /etc/hostname
2024-04-17T14:48:15.688 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088
2024-04-17T14:48:15.754 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release
2024-04-17T14:48:15.809 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream"
2024-04-17T14:48:15.809 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-17T14:48:15.810 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-17T14:48:15.811 INFO:teuthology.provision.fog.smithi088:Deploy complete!
2024-04-17T14:48:15.811 DEBUG:teuthology.parallel:result is None
2024-04-17T14:48:16.022 INFO:teuthology.lock.ops:Checking smithi088.front.sepia.ceph.com
2024-04-17T14:48:16.022 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-17T14:48:16.042 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com
2024-04-17T14:48:16.042 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-17T14:48:16.058 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com
2024-04-17T14:48:16.058 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-17T14:48:16.076 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: set os type and version on server
2024-04-17T14:48:16.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com'
2024-04-17T14:48:16.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:48:16.163 DEBUG:teuthology.orchestra.run.smithi088:> true
2024-04-17T14:48:16.228 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com'
2024-04-17T14:48:16.228 DEBUG:teuthology.orchestra.run.smithi088:> uname -m
2024-04-17T14:48:16.282 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64
2024-04-17T14:48:16.282 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream"
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9"
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos"
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora"
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9"
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9"
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31"
2024-04-17T14:48:16.336 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon"
2024-04-17T14:48:16.337 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-17T14:48:16.337 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/"
2024-04-17T14:48:16.337 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-17T14:48:16.337 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-17T14:48:16.337 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-17T14:48:16.337 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server
2024-04-17T14:48:16.359 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server
2024-04-17T14:48:16.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com'
2024-04-17T14:48:16.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:48:16.447 DEBUG:teuthology.orchestra.run.smithi112:> true
2024-04-17T14:48:16.612 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com'
2024-04-17T14:48:16.613 DEBUG:teuthology.orchestra.run.smithi112:> uname -m
2024-04-17T14:48:16.667 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64
2024-04-17T14:48:16.667 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon"
2024-04-17T14:48:16.722 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-17T14:48:16.723 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/"
2024-04-17T14:48:16.723 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-17T14:48:16.723 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-17T14:48:16.723 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-17T14:48:16.723 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server
2024-04-17T14:48:16.866 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server
2024-04-17T14:48:16.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com'
2024-04-17T14:48:16.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-17T14:48:16.953 DEBUG:teuthology.orchestra.run.smithi161:> true
2024-04-17T14:48:17.116 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com'
2024-04-17T14:48:17.116 DEBUG:teuthology.orchestra.run.smithi161:> uname -m
2024-04-17T14:48:17.170 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64
2024-04-17T14:48:17.171 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release
2024-04-17T14:48:17.224 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream"
2024-04-17T14:48:17.224 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-17T14:48:17.225 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-17T14:48:17.226 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2024-04-17T14:48:17.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-17T14:48:17.357 INFO:teuthology.dispatcher.supervisor:Running job 7660393
2024-04-17T14:48:17.357 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-17_11:26:41-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660393 --name rishabh-2024-04-17_11:26:41-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} -- /home/teuthworker/archive/rishabh-2024-04-17_11:26:41-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660393/orig.config.yaml
2024-04-17T14:48:17.361 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2024-04-17_11:26:41-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660393
2024-04-17T14:48:17.361 INFO:teuthology.dispatcher.supervisor:Job PID: 1517026
2024-04-17T14:48:17.361 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-04-17T14:50:17.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-17T14:52:17.634 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-17T14:54:17.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-17T14:56:17.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-17T14:56:18.059 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1