2022-01-20T05:48:20.632 INFO:root:teuthology version: 1.1.0-75f1bcd2 2022-01-20T05:48:20.804 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging 2022-01-20T05:48:20.805 INFO:teuthology.lock.ops:Updating [ubuntu@smithi138.front.sepia.ceph.com]: reset os type and version on server 2022-01-20T05:48:20.805 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-01-20T05:48:20.855 INFO:teuthology.lock.ops:Node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging is complete 2022-01-20T05:48:20.855 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi192.front.sepia.ceph.com' reimaging 2022-01-20T05:48:20.856 INFO:teuthology.lock.ops:Updating [ubuntu@smithi192.front.sepia.ceph.com]: reset os type and version on server 2022-01-20T05:48:20.856 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2022-01-20T05:48:20.876 INFO:teuthology.lock.ops:Node 'ubuntu@smithi192.front.sepia.ceph.com' reimaging is complete 2022-01-20T05:48:21.075 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of ubuntu 20.04 2022-01-20T05:48:21.080 INFO:teuthology.provision.fog.smithi192:Scheduling deploy of ubuntu 20.04 2022-01-20T05:48:21.451 INFO:teuthology.orchestra.console:Power off smithi138 2022-01-20T05:48:21.489 INFO:teuthology.orchestra.console:Power off smithi192 2022-01-20T05:48:21.619 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-01-20T05:48:21.627 INFO:teuthology.orchestra.console:Power off for smithi192 completed 2022-01-20T05:48:21.721 INFO:teuthology.orchestra.console:Power on smithi138 2022-01-20T05:48:21.729 INFO:teuthology.orchestra.console:Power on smithi192 2022-01-20T05:48:25.982 INFO:teuthology.orchestra.console:Power on for smithi138 completed 2022-01-20T05:48:26.084 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2022-01-20T05:48:30.097 INFO:teuthology.orchestra.console:Power on for smithi192 completed 2022-01-20T05:48:30.199 INFO:teuthology.provision.fog.smithi192:Waiting for deploy to finish 2022-01-20T05:53:33.330 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-20T05:53:33.870 INFO:teuthology.provision.fog.smithi192:Node is ready 2022-01-20T05:53:33.918 INFO:teuthology.orchestra.run.smithi192.stdout:smithi192.front.sepia.ceph.com 2022-01-20T05:53:33.925 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.15.192 smithi192.front.sepia.ceph.com smithi192 2022-01-20T05:53:34.196 INFO:teuthology.provision.fog.smithi192:Deploy complete! 2022-01-20T05:53:34.458 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-20T05:53:34.979 INFO:teuthology.provision.fog.smithi138:Node is ready 2022-01-20T05:53:34.985 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2022-01-20T05:53:35.037 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2022-01-20T05:53:35.241 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2022-01-20T05:53:35.420 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2022-01-20T05:53:35.422 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-20T05:53:35.442 INFO:teuthology.lock.ops:Checking smithi192.front.sepia.ceph.com 2022-01-20T05:53:35.442 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-20T05:53:35.464 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2022-01-20T05:53:35.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-01-20T05:53:35.645 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2022-01-20T05:53:35.651 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="Ubuntu" 2022-01-20T05:53:35.651 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-20T05:53:35.652 INFO:teuthology.orchestra.run.smithi138.stdout:ID=ubuntu 2022-01-20T05:53:35.652 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE=debian 2022-01-20T05:53:35.652 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-20T05:53:35.652 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="20.04" 2022-01-20T05:53:35.653 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-20T05:53:35.653 INFO:teuthology.orchestra.run.smithi138.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-20T05:53:35.653 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-20T05:53:35.653 INFO:teuthology.orchestra.run.smithi138.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-20T05:53:35.654 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_CODENAME=focal 2022-01-20T05:53:35.654 INFO:teuthology.orchestra.run.smithi138.stdout:UBUNTU_CODENAME=focal 2022-01-20T05:53:35.655 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-01-20T05:53:35.674 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: set os type and version on server 2022-01-20T05:53:35.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2022-01-20T05:53:35.843 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2022-01-20T05:53:35.851 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="Ubuntu" 2022-01-20T05:53:35.851 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-20T05:53:35.851 INFO:teuthology.orchestra.run.smithi192.stdout:ID=ubuntu 2022-01-20T05:53:35.851 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE=debian 2022-01-20T05:53:35.852 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-20T05:53:35.852 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="20.04" 2022-01-20T05:53:35.852 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-20T05:53:35.852 INFO:teuthology.orchestra.run.smithi192.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-20T05:53:35.853 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-20T05:53:35.853 INFO:teuthology.orchestra.run.smithi192.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-20T05:53:35.853 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_CODENAME=focal 2022-01-20T05:53:35.853 INFO:teuthology.orchestra.run.smithi192.stdout:UBUNTU_CODENAME=focal 2022-01-20T05:53:35.854 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2022-01-20T05:53:35.952 INFO:teuthology.dispatcher.supervisor:Running job 6628468 2022-01-20T05:53:35.952 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_75f1bcd237a8b6022305cd53ba87e106d45aa665/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-01-20_05:07:02-rgw-pacific-distro-default-smithi/6628468 --name teuthology-2022-01-20_05:07:02-rgw-pacific-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} -- /home/teuthworker/archive/teuthology-2022-01-20_05:07:02-rgw-pacific-distro-default-smithi/6628468/orig.config.yaml 2022-01-20T05:53:35.957 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-01-20_05:07:02-rgw-pacific-distro-default-smithi/6628468 2022-01-20T05:53:35.957 INFO:teuthology.dispatcher.supervisor:Job PID: 11116 2022-01-20T05:53:35.958 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-20T12:24:04.387 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-20T12:24:04.563 INFO:teuthology.kill:Killing Pids: {11116} 2022-01-20T12:24:05.810 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-01-20T12:24:05.811 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-01-20T12:24:05.812 INFO:teuthology.misc:Compressing logs... 2022-01-20T12:24:05.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-01-20T12:24:05.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2022-01-20T12:24:06.289 INFO:teuthology.misc:Compressing logs... 2022-01-20T12:24:06.408 INFO:teuthology.misc:Compressing logs... 2022-01-20T12:24:13.668 INFO:teuthology.kill:No teuthology processes running 2022-01-20T12:24:14.814 INFO:teuthology.kill:Nuking machines: ['smithi138', 'smithi192'] 2022-01-20T12:24:15.778 INFO:teuthology.kill:2022-01-20 12:24:15,777.777 INFO:teuthology.nuke:targets: 2022-01-20T12:24:15.779 INFO:teuthology.kill: smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5PdWVcAKvh/tAz2zzahDM0g6Px65iFQCRw9x2Ky5ycfE/fLKs1Bx5CFjpEDxHdKK8s/ujT9dB6NYvWiGJ46cgOo3WqvGn8JJykTtUuZVY6pkVN5Z/nF9uX8eWNxGLBXB81QL6XlYhXwkWemQz9QXF5TDfs6tz0qHn+T8s5numa+tOwJiOoDkcMPY0x2TW6y8v7k94ptBn3NgHbEDqGOpSL2wtLpIUD6w4h0iSmRFuxZj2JPCsozMi7y2Hmt1+6UrePpmVnxSuaEbS7pU0/oXpudpwZhzLrIwlC0r9U+Cat/1/dFjrfbwVFAXQAxAgR7iIwjtV8IH/1VQSowRLoyulwN3aKAAxxQk4BK6hteHxyhAJJgMuZSomQpRsoQoTbmPItV+umW9PxEhhcqnWm3rpiOd4DQrxpHk8WtmxGcyp2cLJciP8vwhqQfpnpl54C4IgW8ZD9kAS6iWNizsYO4L/1sll0VWiN7K5ulCmNzMursECV8UoBLzBOImeiuL5eU8= 2022-01-20T12:24:15.779 INFO:teuthology.kill: smithi192.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwi7ZYj0pGDPBuKt4xt7skM/Skdz0oE9PXniwoy1sglznMdSRm0//x8UVzAJzY+HcvgryX5e/6+rW95VdukbeVZU0MrHTx94r2wLDGimTnUThSyck5ZlTKc8AmxoI4C0lY841+Auul2WmO0HUd8dz1Psvnw+CLOYAheXQ4pFnGmqhUv32RGrBjWO4P0wVbqOCHy77s/jCAncGRKEF8FqvfnPij6E8ohseMjs8limVT8vsn/MJMqM1N/M7M69f2MRaoqZ9sW/yEd51QxQss1ipPIK1owdk/D9dk0flsB+BsHHQdwREMN3evFgHOk0rTXbJOYAP6LQy33HKjyE+0m5QRtlAuOyIZhihA1S1/PXh/fDZO/gIf1kNUwcr++L7pfte4ZDIQ0+RbzTiY+4Egxi1hD2UYgZBES8rZ01832VaDudYwj1iW40H+7VVfLUxq3T4ujqK2XpQxbT+TNX0HTtXEOhhHbUAo6BA9PacB3BCk2fUYmILfPdmitajngRikIc8= 2022-01-20T12:24:15.805 INFO:teuthology.kill:2022-01-20 12:24:15,805.805 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-20T12:24:15.808 INFO:teuthology.kill:2022-01-20 12:24:15,808.808 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-20T12:24:15.889 INFO:teuthology.kill:2022-01-20 12:24:15,889.889 INFO:teuthology.orchestra.console:Power off smithi192 2022-01-20T12:24:15.904 INFO:teuthology.kill:2022-01-20 12:24:15,903.903 INFO:teuthology.orchestra.console:Power off smithi138 2022-01-20T12:24:28.379 INFO:teuthology.kill:2022-01-20 12:24:28,378.378 INFO:teuthology.orchestra.console:Power off for smithi192 completed 2022-01-20T12:24:28.384 INFO:teuthology.kill:2022-01-20 12:24:28,384.384 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-01-20T12:24:28.529 INFO:teuthology.kill:2022-01-20 12:24:28,528.528 INFO:teuthology.lock.ops:unlocked: smithi192.front.sepia.ceph.com 2022-01-20T12:24:28.531 INFO:teuthology.kill:2022-01-20 12:24:28,531.531 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com 2022-01-20T12:26:28.854 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-20T12:26:28.951 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi138 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-18_05:15:02-krbd-octopus-testing-default-smithi/6623681 2022-01-20T12:26:28.951 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi192 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-18_05:15:02-krbd-octopus-testing-default-smithi/6623681