2024-03-07T13:10:05.150 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-07T13:10:05.150 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:10:05.301 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f9d66ff7f70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f9d68058c10>, signals=[15])
2024-03-07T13:10:05.302 INFO:teuthology.lock.ops:Start node 'smithi022.front.sepia.ceph.com' reimaging
2024-03-07T13:10:05.302 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: reset os type and version on server
2024-03-07T13:10:05.302 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2024-03-07T13:10:05.322 INFO:teuthology.lock.ops:Start node 'smithi103.front.sepia.ceph.com' reimaging
2024-03-07T13:10:05.323 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: reset os type and version on server
2024-03-07T13:10:05.323 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server
2024-03-07T13:10:05.449 DEBUG:teuthology.provision.fog.smithi022:Requesting image smithi_centos_9.stream (ID 10)
2024-03-07T13:10:05.461 DEBUG:teuthology.provision.fog.smithi103:Requesting image smithi_centos_9.stream (ID 10)
2024-03-07T13:10:05.497 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of centos 9.stream
2024-03-07T13:10:05.502 INFO:teuthology.provision.fog.smithi103:Scheduling deploy of centos 9.stream
2024-03-07T13:10:05.719 INFO:teuthology.orchestra.console.smithi103:Power off
2024-03-07T13:10:05.720 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T13:10:05.734 INFO:teuthology.orchestra.console.smithi022:Power off
2024-03-07T13:10:05.734 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T13:10:05.747 DEBUG:teuthology.orchestra.console.smithi103:power off output: Chassis Power Control: Down/Off
2024-03-07T13:10:05.747 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:05.759 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off
2024-03-07T13:10:05.760 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:05.770 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on
2024-03-07T13:10:05.784 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on
2024-03-07T13:10:09.776 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:09.785 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:09.887 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on
2024-03-07T13:10:09.895 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on
2024-03-07T13:10:13.892 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:13.899 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:14.001 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on
2024-03-07T13:10:14.010 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-03-07T13:10:14.110 INFO:teuthology.orchestra.console.smithi022:Power off completed
2024-03-07T13:10:14.211 INFO:teuthology.orchestra.console.smithi022:Power on
2024-03-07T13:10:14.211 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-07T13:10:14.233 DEBUG:teuthology.orchestra.console.smithi022:power on output: Chassis Power Control: Up/On
2024-03-07T13:10:14.234 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:14.259 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-03-07T13:10:18.004 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:18.113 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off
2024-03-07T13:10:18.213 INFO:teuthology.orchestra.console.smithi103:Power off completed
2024-03-07T13:10:18.260 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:18.315 INFO:teuthology.orchestra.console.smithi103:Power on
2024-03-07T13:10:18.315 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-07T13:10:18.335 DEBUG:teuthology.orchestra.console.smithi103:power on output: Chassis Power Control: Up/On
2024-03-07T13:10:18.336 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:18.357 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off
2024-03-07T13:10:18.372 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-03-07T13:10:22.360 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:22.372 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:22.469 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off
2024-03-07T13:10:22.481 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on
2024-03-07T13:10:22.582 INFO:teuthology.orchestra.console.smithi022:Power on completed
2024-03-07T13:10:22.683 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish
2024-03-07T13:10:26.473 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:10:26.583 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on
2024-03-07T13:10:26.684 INFO:teuthology.orchestra.console.smithi103:Power on completed
2024-03-07T13:10:26.784 INFO:teuthology.provision.fog.smithi103:Waiting for deploy to finish
2024-03-07T13:12:38.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:12:38.506 ERROR:teuthology.orchestra.connection:Error authenticating with smithi022.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T13:13:12.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:13:15.548 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:13:21.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:13:24.607 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:13:30.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:13:33.660 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:13:39.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:13:42.508 WARNING:teuthology.provision.fog.smithi022:timed out
2024-03-07T13:13:42.716 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:13:48.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:13:48.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:13:51.772 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:13:57.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:14:00.832 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:14:06.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:14:07.036 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-03-07T13:14:09.884 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:14:13.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:14:13.329 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-07T13:14:13.329 DEBUG:teuthology.orchestra.run.smithi022:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-07T13:14:15.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:14:18.940 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:14:24.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:14:28.441 INFO:teuthology.provision.fog.smithi022:Node is ready
2024-03-07T13:14:28.442 DEBUG:teuthology.orchestra.run.smithi022:> hostname
2024-03-07T13:14:28.461 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com
2024-03-07T13:14:28.461 DEBUG:teuthology.orchestra.run.smithi022:> grep smithi022.front.sepia.ceph.com /etc/hosts
2024-03-07T13:14:28.519 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2024-03-07T13:14:28.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022
2024-03-07T13:14:28.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hosts
2024-03-07T13:14:28.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/172.21.15.22/172.21.15.22/g' /etc/hosts
2024-03-07T13:14:28.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hostname
2024-03-07T13:14:28.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022
2024-03-07T13:14:28.912 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release
2024-03-07T13:14:28.967 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream"
2024-03-07T13:14:28.967 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9"
2024-03-07T13:14:28.967 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos"
2024-03-07T13:14:28.967 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T13:14:28.968 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T13:14:28.969 INFO:teuthology.provision.fog.smithi022:Deploy complete!
2024-03-07T13:14:28.970 DEBUG:teuthology.parallel:result is None
2024-03-07T13:14:43.388 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:14:49.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:14:52.444 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:14:58.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:15:01.500 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103
2024-03-07T13:15:07.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:15:07.806 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-07T13:15:07.806 DEBUG:teuthology.orchestra.run.smithi103:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-07T13:15:23.256 INFO:teuthology.provision.fog.smithi103:Node is ready
2024-03-07T13:15:23.256 DEBUG:teuthology.orchestra.run.smithi103:> hostname
2024-03-07T13:15:23.275 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103.front.sepia.ceph.com
2024-03-07T13:15:23.275 DEBUG:teuthology.orchestra.run.smithi103:> grep smithi103.front.sepia.ceph.com /etc/hosts
2024-03-07T13:15:23.332 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103 smithi103.front.sepia.ceph.com smithi103
2024-03-07T13:15:23.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname smithi103
2024-03-07T13:15:23.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/smithi103.front.sepia.ceph.com/smithi103/g' /etc/hosts
2024-03-07T13:15:23.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/172.21.15.103/172.21.15.103/g' /etc/hosts
2024-03-07T13:15:23.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/smithi103.front.sepia.ceph.com/smithi103/g' /etc/hostname
2024-03-07T13:15:23.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname smithi103
2024-03-07T13:15:23.684 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T13:15:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/"
2024-03-07T13:15:23.740 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T13:15:23.740 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T13:15:23.740 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T13:15:23.740 INFO:teuthology.provision.fog.smithi103:Deploy complete!
2024-03-07T13:15:23.740 DEBUG:teuthology.parallel:result is None
2024-03-07T13:15:23.915 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com
2024-03-07T13:15:23.916 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-07T13:15:23.945 INFO:teuthology.lock.ops:Checking smithi103.front.sepia.ceph.com
2024-03-07T13:15:23.945 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-07T13:15:23.970 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server
2024-03-07T13:15:23.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com'
2024-03-07T13:15:23.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:15:24.077 DEBUG:teuthology.orchestra.run.smithi022:> true
2024-03-07T13:15:24.259 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com'
2024-03-07T13:15:24.259 DEBUG:teuthology.orchestra.run.smithi022:> uname -m
2024-03-07T13:15:24.314 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64
2024-03-07T13:15:24.314 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release
2024-03-07T13:15:24.369 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T13:15:24.370 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T13:15:24.371 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2024-03-07T13:15:24.394 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: set os type and version on server
2024-03-07T13:15:24.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com'
2024-03-07T13:15:24.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:15:24.485 DEBUG:teuthology.orchestra.run.smithi103:> true
2024-03-07T13:15:24.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com'
2024-03-07T13:15:24.556 DEBUG:teuthology.orchestra.run.smithi103:> uname -m
2024-03-07T13:15:24.612 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64
2024-03-07T13:15:24.612 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release
2024-03-07T13:15:24.671 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream"
2024-03-07T13:15:24.671 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9"
2024-03-07T13:15:24.671 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos"
2024-03-07T13:15:24.671 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T13:15:24.672 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T13:15:24.673 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server
2024-03-07T13:15:24.695 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:15:24.785 INFO:teuthology.dispatcher.supervisor:Running job 7584457
2024-03-07T13:15:24.786 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7584457 --name vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/dbench validater/valgrind} -- /home/teuthworker/archive/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7584457/orig.config.yaml
2024-03-07T13:15:24.789 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7584457
2024-03-07T13:15:24.789 INFO:teuthology.dispatcher.supervisor:Job PID: 563977
2024-03-07T13:15:24.789 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-07T13:17:24.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:19:24.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:21:24.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:23:25.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:25:25.078 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:27:25.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:29:25.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:31:25.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:33:25.387 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:35:25.469 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:37:25.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:39:25.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:41:25.717 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:43:25.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:45:25.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:47:25.930 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:49:26.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:50:26.086 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 504: <html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:50:59.432 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:02.200 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:05.850 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:10.390 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:15.803 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:22.099 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:29.277 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:37.350 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:46.291 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:46.291 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 335, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7584457' reached maximum tries (11) after waiting for 49.689975488172784 seconds
2024-03-07T13:53:46.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T19:10:14.823 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T19:10:14.904 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1