2022-01-28T14:26:18.584 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-28T14:26:18.723 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi033.front.sepia.ceph.com' reimaging 2022-01-28T14:26:18.724 INFO:teuthology.lock.ops:Updating [ubuntu@smithi033.front.sepia.ceph.com]: reset os type and version on server 2022-01-28T14:26:18.724 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2022-01-28T14:26:18.743 INFO:teuthology.lock.ops:Node 'ubuntu@smithi033.front.sepia.ceph.com' reimaging is complete 2022-01-28T14:26:18.744 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging 2022-01-28T14:26:18.744 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server 2022-01-28T14:26:18.744 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2022-01-28T14:26:18.762 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete 2022-01-28T14:26:18.965 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of ubuntu 20.04 2022-01-28T14:26:18.983 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of ubuntu 20.04 2022-01-28T14:26:19.313 INFO:teuthology.orchestra.console:Power off smithi116 2022-01-28T14:26:19.327 INFO:teuthology.orchestra.console:Power off smithi033 2022-01-28T14:26:27.684 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2022-01-28T14:26:27.785 INFO:teuthology.orchestra.console:Power on smithi116 2022-01-28T14:26:31.803 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2022-01-28T14:26:31.905 INFO:teuthology.orchestra.console:Power on smithi033 2022-01-28T14:26:36.150 INFO:teuthology.orchestra.console:Power on for smithi116 completed 2022-01-28T14:26:36.251 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2022-01-28T14:26:40.268 INFO:teuthology.orchestra.console:Power on for smithi033 completed 2022-01-28T14:26:40.494 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2022-01-28T14:31:24.537 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-28T14:31:25.096 INFO:teuthology.provision.fog.smithi033:Node is ready 2022-01-28T14:31:25.103 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com 2022-01-28T14:31:25.110 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033 2022-01-28T14:31:25.328 INFO:teuthology.provision.fog.smithi033:Deploy complete! 2022-01-28T14:31:43.863 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-28T14:31:44.464 INFO:teuthology.provision.fog.smithi116:Node is ready 2022-01-28T14:31:44.471 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2022-01-28T14:31:44.478 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2022-01-28T14:31:44.699 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2022-01-28T14:31:44.884 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com 2022-01-28T14:31:44.885 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-28T14:31:44.906 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2022-01-28T14:31:44.906 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-28T14:31:44.924 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server 2022-01-28T14:31:44.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2022-01-28T14:31:45.190 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2022-01-28T14:31:45.197 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Ubuntu" 2022-01-28T14:31:45.197 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-28T14:31:45.198 INFO:teuthology.orchestra.run.smithi033.stdout:ID=ubuntu 2022-01-28T14:31:45.198 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE=debian 2022-01-28T14:31:45.198 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-28T14:31:45.199 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="20.04" 2022-01-28T14:31:45.199 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-28T14:31:45.199 INFO:teuthology.orchestra.run.smithi033.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-28T14:31:45.199 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-28T14:31:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-28T14:31:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_CODENAME=focal 2022-01-28T14:31:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:UBUNTU_CODENAME=focal 2022-01-28T14:31:45.201 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2022-01-28T14:31:45.221 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2022-01-28T14:31:45.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2022-01-28T14:31:45.387 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2022-01-28T14:31:45.393 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Ubuntu" 2022-01-28T14:31:45.394 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-28T14:31:45.394 INFO:teuthology.orchestra.run.smithi116.stdout:ID=ubuntu 2022-01-28T14:31:45.394 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE=debian 2022-01-28T14:31:45.394 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-28T14:31:45.395 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="20.04" 2022-01-28T14:31:45.395 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-28T14:31:45.395 INFO:teuthology.orchestra.run.smithi116.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-28T14:31:45.395 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-28T14:31:45.396 INFO:teuthology.orchestra.run.smithi116.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-28T14:31:45.396 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_CODENAME=focal 2022-01-28T14:31:45.396 INFO:teuthology.orchestra.run.smithi116.stdout:UBUNTU_CODENAME=focal 2022-01-28T14:31:45.396 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2022-01-28T14:31:45.496 INFO:teuthology.dispatcher.supervisor:Running job 6646376 2022-01-28T14:31:45.496 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-01-28_14:22:02-upgrade:octopus-x-quincy-distro-default-smithi/6646376 --name teuthology-2022-01-28_14:22:02-upgrade:octopus-x-quincy-distro-default-smithi --description upgrade:octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/primary} -- /home/teuthworker/archive/teuthology-2022-01-28_14:22:02-upgrade:octopus-x-quincy-distro-default-smithi/6646376/orig.config.yaml 2022-01-28T14:31:45.500 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-01-28_14:22:02-upgrade:octopus-x-quincy-distro-default-smithi/6646376 2022-01-28T14:31:45.500 INFO:teuthology.dispatcher.supervisor:Job PID: 27402 2022-01-28T14:31:45.501 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-28T21:02:14.088 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-28T21:02:14.320 INFO:teuthology.kill:Killing Pids: {27402} 2022-01-28T21:02:15.497 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0'] 2022-01-28T21:02:15.498 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-01-28T21:02:15.498 INFO:teuthology.misc:Compressing logs... 2022-01-28T21:02:15.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2022-01-28T21:02:15.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2022-01-28T21:02:16.268 INFO:teuthology.misc:Compressing logs... 2022-01-28T21:02:16.390 INFO:teuthology.misc:Compressing logs... 2022-01-28T21:15:39.481 INFO:teuthology.kill:No teuthology processes running 2022-01-28T21:15:40.663 INFO:teuthology.kill:Nuking machines: ['smithi033', 'smithi116'] 2022-01-28T21:15:41.671 INFO:teuthology.kill:2022-01-28 21:15:41,671.671 INFO:teuthology.nuke:targets: 2022-01-28T21:15:41.672 INFO:teuthology.kill: smithi033.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwZpSfha5HcD9Y0QczZreg3WAuQuMvrkcSUl0CtIfBm9+8NG6AxIoQuUgWydLsN8bw1D7z1tPRhTaMy7Xgqfk3uCC8vwkzRn/7hVlHWA7oyMk7N6Je4sTWjCbinirBQNbLrLqw8+3zjcRMeLAJeEHWQslYhWgAx0UYLejsRO3yFEzu6r7Vx7SsR30eTCUJ63jO0gixAgW+MsJbEANfWXLVf9hZl6bUdImPZlhK2864MimUgJaPIACt9ApVVq2Wd148kq2nYLBuUxbx4CPSNC7sp1AYExHEmtgLP1VyQxaPYuUZn49Osqxb576Mq2smf3EVTiQkMz64fzioJqWXj5Qte01pGBmDb6oHTq3Ggh/auX0aiUMTJGBn9tZsv4M2wXbSTLy3msQGIK6mBfRxIj6X3i+3LZmI90kL0KZUXUc8FAXUbLpG41jTD+C2AGOgN7G/o8rBVf5fyhhLZ/wiDstslg4AVYQxVzn3ZDq/2X40LSzS+btgxXBaSXYyi+8NnNE= 2022-01-28T21:15:41.673 INFO:teuthology.kill: smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9oYD8xI7xgxfNZoU3JKu5Nkdz7T8+Fppnt/tYbRBVKprfw92/5HNH5eErP2UaWzjKqZgJZreyGHImqbF2vY3/3ZjBC7vcKgjWofR0aWnRBtgASlRkA/qjQW4rhNsFa05PblEDm32atNJkDOnjK1/vNcuLGuhVRpcNPlUwsI0yYEqSWiuou3r0KhEQNONZJklRjFAuVMdvlR4qT+lS+iWqrXgSGjCV6TPjBEe3yg/PT/22kFEpHUnHNJknuaTmq9fFRF0MwBSANNNYB441XkK5KEGZzhZMjo30OP2QDNHsbLdYlbC5noFhc6u+oIMJAflht1ZhPy2hLKSOXtfN7Hi2LT2oISXgSdZ5LF5v6tJSDlo/KZXpFmulVQJrDEoz3Wf9vsg5BccuS1bsThEl8yAKGpxiL/flVZvVRAldRmNDxxoX0q1FaEQwnqYGtAIv8vINL3/XK1mnCpTp/TjDbuY0fYcFncCipPMYrHaiI39fcjhjJRbbz/teDlFH4Z7j6fc= 2022-01-28T21:15:41.699 INFO:teuthology.kill:2022-01-28 21:15:41,699.699 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-28T21:15:41.701 INFO:teuthology.kill:2022-01-28 21:15:41,701.701 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-28T21:15:41.777 INFO:teuthology.kill:2022-01-28 21:15:41,776.776 INFO:teuthology.orchestra.console:Power off smithi033 2022-01-28T21:15:41.790 INFO:teuthology.kill:2022-01-28 21:15:41,790.790 INFO:teuthology.orchestra.console:Power off smithi116 2022-01-28T21:15:50.160 INFO:teuthology.kill:2022-01-28 21:15:50,159.159 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2022-01-28T21:15:50.307 INFO:teuthology.kill:2022-01-28 21:15:50,306.306 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com 2022-01-28T21:15:54.256 INFO:teuthology.kill:2022-01-28 21:15:54,255.255 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2022-01-28T21:15:54.400 INFO:teuthology.kill:2022-01-28 21:15:54,399.399 INFO:teuthology.lock.ops:unlocked: smithi033.front.sepia.ceph.com 2022-01-28T21:17:54.706 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-28T21:17:54.802 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi033 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-28_16:09:05-rados-wip-master-1.27.22-distro-default-smithi/6646650 2022-01-28T21:17:54.803 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi116 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-28_16:09:05-rados-wip-master-1.27.22-distro-default-smithi/6646649