2021-04-29T05:16:02.818 INFO:root:teuthology version: 1.1.0-bea5b739 2021-04-29T05:16:02.916 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi133.front.sepia.ceph.com' reimaging 2021-04-29T05:16:02.917 INFO:teuthology.lock.ops:Updating [ubuntu@smithi133.front.sepia.ceph.com]: reset os type and version on server 2021-04-29T05:16:02.917 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2021-04-29T05:16:02.934 INFO:teuthology.lock.ops:Node 'ubuntu@smithi133.front.sepia.ceph.com' reimaging is complete 2021-04-29T05:16:02.934 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi173.front.sepia.ceph.com' reimaging 2021-04-29T05:16:02.934 INFO:teuthology.lock.ops:Updating [ubuntu@smithi173.front.sepia.ceph.com]: reset os type and version on server 2021-04-29T05:16:02.934 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2021-04-29T05:16:02.949 INFO:teuthology.lock.ops:Node 'ubuntu@smithi173.front.sepia.ceph.com' reimaging is complete 2021-04-29T05:16:03.148 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of ubuntu 20.04 2021-04-29T05:16:03.160 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of ubuntu 20.04 2021-04-29T05:16:03.466 INFO:teuthology.orchestra.console:Power off smithi133 2021-04-29T05:16:03.483 INFO:teuthology.orchestra.console:Power off smithi173 2021-04-29T05:16:11.862 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2021-04-29T05:16:11.868 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2021-04-29T05:16:11.963 INFO:teuthology.orchestra.console:Power on smithi133 2021-04-29T05:16:11.970 INFO:teuthology.orchestra.console:Power on smithi173 2021-04-29T05:16:16.217 INFO:teuthology.orchestra.console:Power on for smithi133 completed 2021-04-29T05:16:16.318 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish 2021-04-29T05:16:20.437 INFO:teuthology.orchestra.console:Power on for smithi173 completed 2021-04-29T05:16:20.538 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2021-04-29T05:21:14.738 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-29T05:21:15.199 INFO:teuthology.provision.fog.smithi173:Node is ready 2021-04-29T05:21:15.247 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com 2021-04-29T05:21:15.254 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173 2021-04-29T05:21:15.451 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2021-04-29T05:21:23.841 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-29T05:21:24.317 INFO:teuthology.provision.fog.smithi133:Node is ready 2021-04-29T05:21:24.324 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com 2021-04-29T05:21:24.330 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133 2021-04-29T05:21:24.519 INFO:teuthology.provision.fog.smithi133:Deploy complete! 2021-04-29T05:21:25.754 INFO:teuthology.lock.ops:Checking smithi133.front.sepia.ceph.com 2021-04-29T05:21:25.754 INFO:teuthology.lock.ops:New key found. Updating... 2021-04-29T05:21:26.201 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2021-04-29T05:21:26.201 INFO:teuthology.lock.ops:New key found. Updating... 2021-04-29T05:21:26.258 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: set os type and version on server 2021-04-29T05:21:26.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-29T05:21:26.426 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2021-04-29T05:21:26.433 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Ubuntu" 2021-04-29T05:21:26.433 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-04-29T05:21:26.433 INFO:teuthology.orchestra.run.smithi133.stdout:ID=ubuntu 2021-04-29T05:21:26.433 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE=debian 2021-04-29T05:21:26.433 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-04-29T05:21:26.433 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="20.04" 2021-04-29T05:21:26.433 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.ubuntu.com/" 2021-04-29T05:21:26.434 INFO:teuthology.orchestra.run.smithi133.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-04-29T05:21:26.434 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-04-29T05:21:26.434 INFO:teuthology.orchestra.run.smithi133.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-04-29T05:21:26.434 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_CODENAME=focal 2021-04-29T05:21:26.434 INFO:teuthology.orchestra.run.smithi133.stdout:UBUNTU_CODENAME=focal 2021-04-29T05:21:26.435 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2021-04-29T05:21:26.496 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2021-04-29T05:21:26.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-29T05:21:26.671 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2021-04-29T05:21:26.679 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="Ubuntu" 2021-04-29T05:21:26.679 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-04-29T05:21:26.679 INFO:teuthology.orchestra.run.smithi173.stdout:ID=ubuntu 2021-04-29T05:21:26.679 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE=debian 2021-04-29T05:21:26.679 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-04-29T05:21:26.679 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="20.04" 2021-04-29T05:21:26.679 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.ubuntu.com/" 2021-04-29T05:21:26.680 INFO:teuthology.orchestra.run.smithi173.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-04-29T05:21:26.680 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-04-29T05:21:26.680 INFO:teuthology.orchestra.run.smithi173.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-04-29T05:21:26.680 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_CODENAME=focal 2021-04-29T05:21:26.680 INFO:teuthology.orchestra.run.smithi173.stdout:UBUNTU_CODENAME=focal 2021-04-29T05:21:26.680 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2021-04-29T05:21:26.775 INFO:teuthology.dispatcher.supervisor:Running job 6081160 2021-04-29T05:21:26.776 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/virtualenv/bin/teuthology -v --owner scheduled_dis@teuthology --archive /home/teuthworker/archive/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/6081160 --name dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi --description rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{ubuntu_latest} 4-pool/cache 5-cache-mode/ssd 6-workloads/fio} -- /home/teuthworker/archive/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/6081160/orig.config.yaml 2021-04-29T05:21:26.780 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/6081160 2021-04-29T05:21:26.780 INFO:teuthology.dispatcher.supervisor:Job PID: 15067 2021-04-29T05:21:26.780 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-04-29T15:34:54.001 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:34:55.088 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:34:56.637 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:34:58.733 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:35:01.374 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:35:04.558 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:35:08.337 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:35:12.615 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:35:17.438 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:35:22.805 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/dis-2021-04-29_04:40:41-rbd-wip-dis-testing-1-distro-basic-smithi/jobs/ failed with status 502: 502 Bad Gateway

502 Bad Gateway


nginx/1.10.3 (Ubuntu)
2021-04-29T15:35:22.806 ERROR:teuthology.dispatcher.supervisor:run_with_watchdog had an unhandled exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/dispatcher/supervisor.py", line 136, in run_job run_with_watchdog(p, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/dispatcher/supervisor.py", line 243, in run_with_watchdog report.try_push_job_info(job_info) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/report.py", line 505, in try_push_job_info push_job_info(run_name, job_id, job_info) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/report.py", line 470, in push_job_info reporter.report_job(run_name, job_id, job_info) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/report.py", line 298, in report_job while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'report job 6081160' reached maximum tries (10) after waiting for 34.51908883795263 seconds 2021-04-29T15:35:22.807 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/dispatcher/supervisor.py", line 59, in main verbose File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/dispatcher/supervisor.py", line 136, in run_job run_with_watchdog(p, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/dispatcher/supervisor.py", line 243, in run_with_watchdog report.try_push_job_info(job_info) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/report.py", line 505, in try_push_job_info push_job_info(run_name, job_id, job_info) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/report.py", line 470, in push_job_info reporter.report_job(run_name, job_id, job_info) File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/report.py", line 298, in report_job while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'report job 6081160' reached maximum tries (10) after waiting for 34.51908883795263 seconds