2024-01-16T17:35:52.025 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-01-16T17:35:52.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:35:52.168 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f78bab3bf40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f78bbba1c10>, signals=[15])
2024-01-16T17:35:52.168 INFO:teuthology.lock.ops:Start node 'smithi001.front.sepia.ceph.com' reimaging
2024-01-16T17:35:52.169 INFO:teuthology.lock.ops:Updating [smithi001.front.sepia.ceph.com]: reset os type and version on server
2024-01-16T17:35:52.169 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server
2024-01-16T17:35:52.193 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging
2024-01-16T17:35:52.193 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server
2024-01-16T17:35:52.193 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2024-01-16T17:35:52.310 DEBUG:teuthology.provision.fog.smithi001:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-01-16T17:35:52.324 DEBUG:teuthology.provision.fog.smithi119:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-01-16T17:35:52.372 INFO:teuthology.provision.fog.smithi001:Scheduling deploy of ubuntu 22.04
2024-01-16T17:35:52.378 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of ubuntu 22.04
2024-01-16T17:35:52.639 INFO:teuthology.orchestra.console.smithi119:Power off
2024-01-16T17:35:52.640 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-16T17:35:52.652 INFO:teuthology.orchestra.console.smithi001:Power off
2024-01-16T17:35:52.652 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-16T17:35:52.660 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off
2024-01-16T17:35:52.661 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:35:52.673 DEBUG:teuthology.orchestra.console.smithi001:power off output: Chassis Power Control: Down/Off
2024-01-16T17:35:52.673 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:35:52.681 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on
2024-01-16T17:35:52.694 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is off
2024-01-16T17:35:52.795 INFO:teuthology.orchestra.console.smithi001:Power off completed
2024-01-16T17:35:52.895 INFO:teuthology.orchestra.console.smithi001:Power on
2024-01-16T17:35:52.896 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-16T17:35:52.917 DEBUG:teuthology.orchestra.console.smithi001:power on output: Chassis Power Control: Up/On
2024-01-16T17:35:52.917 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:35:52.937 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is off
2024-01-16T17:35:56.686 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:35:56.794 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on
2024-01-16T17:35:56.939 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:35:57.047 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is on
2024-01-16T17:35:57.147 INFO:teuthology.orchestra.console.smithi001:Power on completed
2024-01-16T17:35:57.248 INFO:teuthology.provision.fog.smithi001:Waiting for deploy to finish
2024-01-16T17:36:00.796 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:36:00.904 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on
2024-01-16T17:36:04.908 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:36:05.017 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2024-01-16T17:36:05.118 INFO:teuthology.orchestra.console.smithi119:Power off completed
2024-01-16T17:36:05.218 INFO:teuthology.orchestra.console.smithi119:Power on
2024-01-16T17:36:05.218 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-16T17:36:05.239 DEBUG:teuthology.orchestra.console.smithi119:power on output: Chassis Power Control: Up/On
2024-01-16T17:36:05.239 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:36:05.258 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2024-01-16T17:36:09.262 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-16T17:36:09.371 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on
2024-01-16T17:36:09.472 INFO:teuthology.orchestra.console.smithi119:Power on completed
2024-01-16T17:36:09.573 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish
2024-01-16T17:39:28.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:39:47.388 WARNING:teuthology.provision.fog.smithi001:[Errno None] Unable to connect to port 22 on 172.21.15.1
2024-01-16T17:39:53.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:39:56.476 WARNING:teuthology.provision.fog.smithi001:[Errno None] Unable to connect to port 22 on 172.21.15.1
2024-01-16T17:39:56.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:40:02.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:40:05.564 WARNING:teuthology.provision.fog.smithi001:[Errno None] Unable to connect to port 22 on 172.21.15.1
2024-01-16T17:40:11.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:40:15.036 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-16T17:40:21.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:40:24.125 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-16T17:40:30.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:40:33.180 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-16T17:40:39.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:40:42.271 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-16T17:40:48.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:41:11.599 WARNING:teuthology.provision.fog.smithi001:timed out
2024-01-16T17:41:17.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:41:17.814 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-16T17:41:17.814 DEBUG:teuthology.orchestra.run.smithi001:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-16T17:41:18.288 INFO:teuthology.provision.fog.smithi001:Node is ready
2024-01-16T17:41:18.288 DEBUG:teuthology.orchestra.run.smithi001:> hostname
2024-01-16T17:41:18.335 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001.front.sepia.ceph.com
2024-01-16T17:41:18.335 DEBUG:teuthology.orchestra.run.smithi001:> grep smithi001.front.sepia.ceph.com /etc/hosts
2024-01-16T17:41:18.383 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1 smithi001.front.sepia.ceph.com smithi001
2024-01-16T17:41:18.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname smithi001
2024-01-16T17:41:18.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i -e 's/smithi001.front.sepia.ceph.com/smithi001/g' /etc/hosts
2024-01-16T17:41:18.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i -e 's/172.21.15.1/172.21.15.1/g' /etc/hosts
2024-01-16T17:41:18.615 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i -e 's/smithi001.front.sepia.ceph.com/smithi001/g' /etc/hostname
2024-01-16T17:41:18.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname smithi001
2024-01-16T17:41:18.726 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release
2024-01-16T17:41:18.771 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-01-16T17:41:18.771 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="Ubuntu"
2024-01-16T17:41:18.771 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="22.04"
2024-01-16T17:41:18.771 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-01-16T17:41:18.771 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_CODENAME=jammy
2024-01-16T17:41:18.772 INFO:teuthology.orchestra.run.smithi001.stdout:ID=ubuntu
2024-01-16T17:41:18.772 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE=debian
2024-01-16T17:41:18.772 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-16T17:41:18.772 INFO:teuthology.orchestra.run.smithi001.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-16T17:41:18.772 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-16T17:41:18.772 INFO:teuthology.orchestra.run.smithi001.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-16T17:41:18.772 INFO:teuthology.orchestra.run.smithi001.stdout:UBUNTU_CODENAME=jammy
2024-01-16T17:41:18.772 INFO:teuthology.provision.fog.smithi001:Deploy complete!
2024-01-16T17:41:18.774 DEBUG:teuthology.parallel:result is None
2024-01-16T17:41:48.288 WARNING:teuthology.provision.fog.smithi119:timed out
2024-01-16T17:41:54.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:41:54.483 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-16T17:41:54.483 DEBUG:teuthology.orchestra.run.smithi119:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-16T17:41:54.965 INFO:teuthology.provision.fog.smithi119:Node is ready
2024-01-16T17:41:54.965 DEBUG:teuthology.orchestra.run.smithi119:> hostname
2024-01-16T17:41:55.014 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119.front.sepia.ceph.com
2024-01-16T17:41:55.014 DEBUG:teuthology.orchestra.run.smithi119:> grep smithi119.front.sepia.ceph.com /etc/hosts
2024-01-16T17:41:55.062 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119 smithi119.front.sepia.ceph.com smithi119
2024-01-16T17:41:55.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119
2024-01-16T17:41:55.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hosts
2024-01-16T17:41:55.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/172.21.15.119/172.21.15.119/g' /etc/hosts
2024-01-16T17:41:55.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hostname
2024-01-16T17:41:55.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119
2024-01-16T17:41:55.361 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu"
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="22.04"
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=jammy
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-16T17:41:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=jammy
2024-01-16T17:41:55.407 INFO:teuthology.provision.fog.smithi119:Deploy complete!
2024-01-16T17:41:55.407 DEBUG:teuthology.parallel:result is None
2024-01-16T17:41:55.596 INFO:teuthology.lock.ops:Checking smithi001.front.sepia.ceph.com
2024-01-16T17:41:55.597 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-16T17:41:55.617 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com
2024-01-16T17:41:55.617 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-16T17:41:55.634 INFO:teuthology.lock.ops:Updating [smithi001.front.sepia.ceph.com]: set os type and version on server
2024-01-16T17:41:55.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com'
2024-01-16T17:41:55.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:41:55.712 DEBUG:teuthology.orchestra.run.smithi001:> true
2024-01-16T17:41:55.948 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com'
2024-01-16T17:41:55.948 DEBUG:teuthology.orchestra.run.smithi001:> uname -m
2024-01-16T17:41:55.995 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64
2024-01-16T17:41:55.996 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release
2024-01-16T17:41:56.043 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-01-16T17:41:56.043 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="Ubuntu"
2024-01-16T17:41:56.043 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="22.04"
2024-01-16T17:41:56.043 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-01-16T17:41:56.043 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_CODENAME=jammy
2024-01-16T17:41:56.043 INFO:teuthology.orchestra.run.smithi001.stdout:ID=ubuntu
2024-01-16T17:41:56.043 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE=debian
2024-01-16T17:41:56.044 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-16T17:41:56.044 INFO:teuthology.orchestra.run.smithi001.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-16T17:41:56.044 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-16T17:41:56.044 INFO:teuthology.orchestra.run.smithi001.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-16T17:41:56.044 INFO:teuthology.orchestra.run.smithi001.stdout:UBUNTU_CODENAME=jammy
2024-01-16T17:41:56.044 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server
2024-01-16T17:41:56.067 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server
2024-01-16T17:41:56.067 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com'
2024-01-16T17:41:56.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-16T17:41:56.144 DEBUG:teuthology.orchestra.run.smithi119:> true
2024-01-16T17:41:56.235 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com'
2024-01-16T17:41:56.235 DEBUG:teuthology.orchestra.run.smithi119:> uname -m
2024-01-16T17:41:56.286 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64
2024-01-16T17:41:56.286 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release
2024-01-16T17:41:56.333 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-01-16T17:41:56.333 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu"
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="22.04"
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=jammy
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-16T17:41:56.334 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=jammy
2024-01-16T17:41:56.334 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2024-01-16T17:41:56.358 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:41:56.439 INFO:teuthology.dispatcher.supervisor:Running job 7518306
2024-01-16T17:41:56.439 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/bin/teuthology -v --owner scheduled_rraja@teuthology --archive /home/teuthworker/archive/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/7518306 --name rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi --description rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd} -- /home/teuthworker/archive/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/7518306/orig.config.yaml
2024-01-16T17:41:56.441 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/7518306
2024-01-16T17:41:56.442 INFO:teuthology.dispatcher.supervisor:Job PID: 3081494
2024-01-16T17:41:56.442 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-16T17:43:56.540 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:45:56.720 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:47:56.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:49:57.128 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:51:57.290 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:53:57.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:55:57.618 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:57:57.788 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T17:59:57.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:01:58.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:03:58.196 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:05:58.368 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:07:58.536 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:09:58.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:11:58.880 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:13:59.018 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:15:59.177 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:17:59.340 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:19:59.436 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:21:59.604 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:23:59.734 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:25:59.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:28:00.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:30:00.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:32:00.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:34:00.547 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:36:00.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:38:00.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:40:01.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:42:01.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:44:01.407 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:46:01.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:48:01.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:50:01.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:52:02.081 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:54:02.145 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:56:02.300 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T18:58:02.464 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:00:02.644 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:02:02.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:04:03.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:06:03.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:08:03.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:10:03.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:12:03.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:14:03.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:16:03.969 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:18:04.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:20:04.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:22:04.458 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:24:04.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:26:04.796 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:28:04.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:30:05.107 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:32:05.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:34:05.415 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:36:05.581 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:38:05.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:40:05.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:42:06.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:44:06.244 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:46:06.398 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:46:06.472 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:08.258 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:10.769 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:13.999 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:17.947 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:22.622 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:28.020 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:34.142 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:40.985 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:48.544 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:46:48.544 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 42.46844409343099 seconds
2024-01-16T19:48:48.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:48:48.694 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:48:50.483 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:48:52.997 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:48:56.236 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:49:00.195 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:49:04.880 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:49:10.292 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:49:16.424 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:49:23.293 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:49:30.881 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:49:30.881 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 42.597126710667666 seconds
2024-01-16T19:51:30.980 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:51:31.048 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:51:32.955 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:51:35.704 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:51:39.308 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:51:43.756 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:51:49.049 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:51:55.187 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:52:02.163 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:52:09.985 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:52:18.653 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:52:18.653 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 47.99540675897144 seconds
2024-01-16T19:54:18.690 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:54:18.771 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:54:20.674 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:54:23.419 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:54:26.996 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:54:31.409 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:54:36.667 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:54:42.762 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:54:49.709 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:54:57.491 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:55:06.113 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:55:06.113 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 47.77314854860457 seconds
2024-01-16T19:57:06.196 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:57:06.265 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:07.797 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:09.798 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:12.256 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:15.170 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:18.556 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:22.411 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:26.728 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:31.505 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:36.752 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:57:36.752 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 30.9136340292513 seconds
2024-01-16T19:59:36.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T19:59:36.917 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:59:38.421 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:59:40.366 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:59:42.754 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:59:45.587 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:59:48.865 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:59:52.585 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T19:59:56.754 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:00:01.362 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:00:06.414 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:00:06.414 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 29.947957999854466 seconds
2024-01-16T20:02:06.496 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T20:02:06.569 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:08.518 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:11.373 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:15.115 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:19.749 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:25.282 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:31.711 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:39.023 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:47.234 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:56.331 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:02:56.331 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 50.177803902040075 seconds
2024-01-16T20:04:56.432 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T20:04:56.496 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:04:58.329 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:00.919 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:04.288 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:08.428 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:13.332 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:19.005 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:25.460 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:32.666 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:40.640 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:05:40.641 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 44.56117171118463 seconds
2024-01-16T20:07:40.742 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T20:07:40.810 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:07:42.825 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:07:45.806 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:07:49.749 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:07:54.630 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:08:00.477 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:08:07.282 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:08:15.040 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:08:23.755 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:08:33.419 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:08:33.420 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 52.98178585807125 seconds
2024-01-16T20:10:33.433 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T20:10:33.504 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:34.728 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:36.124 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:37.686 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:39.425 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:41.327 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:43.402 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:45.668 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:48.078 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:50.662 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:10:50.662 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 17.59279495971826 seconds
2024-01-16T20:12:50.756 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T20:12:50.830 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:12:52.063 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:12:53.473 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:12:55.067 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:12:56.828 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:12:58.809 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:13:00.930 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:13:03.223 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:13:05.738 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:13:08.382 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:13:08.383 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 17.935560686439832 seconds
2024-01-16T20:15:08.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T20:15:08.462 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:09.763 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:11.301 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:13.073 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:15.087 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:17.438 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:19.932 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:22.667 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:25.641 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:28.856 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:15:28.856 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 340, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 20.7649150856097 seconds
2024-01-16T20:17:28.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-16T20:17:29.027 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:30.236 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:31.587 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:33.084 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:34.739 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:36.533 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:38.475 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:40.558 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:42.785 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:45.160 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/rraja-2024-01-16_17:34:50-rbd:mirror-main-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2024-01-16T20:17:45.160 ERROR:teuthology.dispatcher.supervisor:run_with_watchdog had an unhandled exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 162, in run_job
    run_with_watchdog(p, job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 357, in run_with_watchdog
    report.try_push_job_info(job_info, extra_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 16.58576419920481 seconds
2024-01-16T20:17:45.161 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 68, in main
    return run_job(
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 162, in run_job
    run_with_watchdog(p, job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 357, in run_with_watchdog
    report.try_push_job_info(job_info, extra_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7518306' reached maximum tries (11) after waiting for 16.58576419920481 seconds