2022-03-10T09:49:35.623 INFO:root:teuthology version: 1.1.0-ff5db916 2022-03-10T09:49:35.744 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi097.front.sepia.ceph.com' reimaging 2022-03-10T09:49:35.745 INFO:teuthology.lock.ops:Updating [ubuntu@smithi097.front.sepia.ceph.com]: reset os type and version on server 2022-03-10T09:49:35.746 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2022-03-10T09:49:35.766 INFO:teuthology.lock.ops:Node 'ubuntu@smithi097.front.sepia.ceph.com' reimaging is complete 2022-03-10T09:49:35.766 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging 2022-03-10T09:49:35.767 INFO:teuthology.lock.ops:Updating [ubuntu@smithi138.front.sepia.ceph.com]: reset os type and version on server 2022-03-10T09:49:35.767 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-03-10T09:49:35.787 INFO:teuthology.lock.ops:Node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging is complete 2022-03-10T09:49:35.980 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of ubuntu 20.04 2022-03-10T09:49:35.999 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of ubuntu 20.04 2022-03-10T09:49:36.389 INFO:teuthology.orchestra.console:Power off smithi097 2022-03-10T09:49:36.429 INFO:teuthology.orchestra.console:Power off smithi138 2022-03-10T09:49:48.879 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2022-03-10T09:49:48.900 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-03-10T09:49:48.982 INFO:teuthology.orchestra.console:Power on smithi097 2022-03-10T09:49:49.001 INFO:teuthology.orchestra.console:Power on smithi138 2022-03-10T09:49:53.252 INFO:teuthology.orchestra.console:Power on for smithi138 completed 2022-03-10T09:49:53.354 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2022-03-10T09:49:57.360 INFO:teuthology.orchestra.console:Power on for smithi097 completed 2022-03-10T09:49:57.461 INFO:teuthology.provision.fog.smithi097:Waiting for deploy to finish 2022-03-10T09:54:58.103 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-10T09:54:58.560 INFO:teuthology.provision.fog.smithi097:Node is ready 2022-03-10T09:54:58.565 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097.front.sepia.ceph.com 2022-03-10T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97 smithi097.front.sepia.ceph.com smithi097 2022-03-10T09:54:58.785 INFO:teuthology.provision.fog.smithi097:Deploy complete! 2022-03-10T09:55:03.020 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-10T09:55:03.430 INFO:teuthology.provision.fog.smithi138:Node is ready 2022-03-10T09:55:03.437 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2022-03-10T09:55:03.444 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2022-03-10T09:55:03.641 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2022-03-10T09:55:03.857 INFO:teuthology.lock.ops:Checking smithi097.front.sepia.ceph.com 2022-03-10T09:55:03.858 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-10T09:55:03.880 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2022-03-10T09:55:03.881 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-10T09:55:03.901 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: set os type and version on server 2022-03-10T09:55:03.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2022-03-10T09:55:04.127 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2022-03-10T09:55:04.134 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Ubuntu" 2022-03-10T09:55:04.135 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-10T09:55:04.135 INFO:teuthology.orchestra.run.smithi097.stdout:ID=ubuntu 2022-03-10T09:55:04.135 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE=debian 2022-03-10T09:55:04.136 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-10T09:55:04.136 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="20.04" 2022-03-10T09:55:04.136 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-10T09:55:04.136 INFO:teuthology.orchestra.run.smithi097.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-10T09:55:04.137 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-10T09:55:04.137 INFO:teuthology.orchestra.run.smithi097.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-10T09:55:04.137 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_CODENAME=focal 2022-03-10T09:55:04.137 INFO:teuthology.orchestra.run.smithi097.stdout:UBUNTU_CODENAME=focal 2022-03-10T09:55:04.138 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2022-03-10T09:55:04.156 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2022-03-10T09:55:04.156 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-03-10T09:55:04.327 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2022-03-10T09:55:04.333 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="Ubuntu" 2022-03-10T09:55:04.334 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-10T09:55:04.334 INFO:teuthology.orchestra.run.smithi138.stdout:ID=ubuntu 2022-03-10T09:55:04.334 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE=debian 2022-03-10T09:55:04.334 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-10T09:55:04.335 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="20.04" 2022-03-10T09:55:04.335 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-10T09:55:04.335 INFO:teuthology.orchestra.run.smithi138.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-10T09:55:04.335 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-10T09:55:04.335 INFO:teuthology.orchestra.run.smithi138.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-10T09:55:04.336 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_CODENAME=focal 2022-03-10T09:55:04.336 INFO:teuthology.orchestra.run.smithi138.stdout:UBUNTU_CODENAME=focal 2022-03-10T09:55:04.336 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-03-10T09:55:04.438 INFO:teuthology.dispatcher.supervisor:Running job 6729584 2022-03-10T09:55:04.438 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_ff5db9169566776b2f41faf671316ca58a318ea6/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-03-10_05:07:01-rgw-pacific-distro-default-smithi/6729584 --name teuthology-2022-03-10_05:07:01-rgw-pacific-distro-default-smithi --description rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} -- /home/teuthworker/archive/teuthology-2022-03-10_05:07:01-rgw-pacific-distro-default-smithi/6729584/orig.config.yaml 2022-03-10T09:55:04.442 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-03-10_05:07:01-rgw-pacific-distro-default-smithi/6729584 2022-03-10T09:55:04.443 INFO:teuthology.dispatcher.supervisor:Job PID: 30805 2022-03-10T09:55:04.443 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-10T16:25:33.551 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-10T16:25:33.684 INFO:teuthology.kill:Killing Pids: {30805} 2022-03-10T16:25:34.886 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-03-10T16:25:34.887 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-03-10T16:25:34.887 INFO:teuthology.misc:Compressing logs... 2022-03-10T16:25:34.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2022-03-10T16:25:35.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-03-10T16:25:35.385 INFO:teuthology.misc:Compressing logs... 2022-03-10T16:25:35.517 INFO:teuthology.misc:Compressing logs... 2022-03-10T16:27:34.236 INFO:teuthology.kill:No teuthology processes running 2022-03-10T16:27:35.528 INFO:teuthology.kill:Nuking machines: ['smithi097', 'smithi138'] 2022-03-10T16:27:36.556 INFO:teuthology.kill:2022-03-10 16:27:36,555.555 INFO:teuthology.nuke:targets: 2022-03-10T16:27:36.557 INFO:teuthology.kill: smithi097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC118CZ6HyflA+jJbmSUOEC4lvRNubZ0facUc/eg6eSzNfYXEvUn6P2vX5Eb0BxBWoC07thbIviM8R5iAOWRbG9aX+4Ewznd1RJ4dmlJL+ZrfVDNcZB9x/+TDifardPNTa+dgy8cDqMOOPOo0vBWrIa+CCOdfUN7Lq9QF4G8ZzKYFVX/7ACr6Y2gEtumGoLQZ5e+4qe6mY4MEs070P4YiCZxa7vwY/LwDYCnGz61csS6vUNPlS28sHqVnTcWLJD5iNks5o5LvuGcaep6e9zrV081ZofwCVt9X2D15Tijv35Cx4+ohoKMwS5Z2E5Y/uYFX1J+T15IQ2LmdAIAUrwZQkgV+rwesjBXgXTUNLoHrO/ugUi1G3yCvU9atZpc/foKBTcNAf9XMEM9e/hLF6KNsEwgCh8MEIHvzGHFCT07uuT9Bn8/LusI/hBSfLBLMaaY/OvYXuqv8W8HoRkFny6nUsh7W9JtTnU8A44xCyhAWq1BkXJoXop4bAbPTiiGgYXln0= 2022-03-10T16:27:36.558 INFO:teuthology.kill: smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDILbXescbTZE+bFdXADf1ZMtLh/hNq1IByf74z0Unb2iPrD9FumbZFEhFzGj4SpMlkqi+na14oQ62+OUOSTA1iDGazUrNs98Wh/fdAZErzgy5BFs/LJR8331r38MSdiRDrkkc6kxs84ILA2JLA4dSzciU1Eb6bGy8ft7dWiAqLpDG3MWXQN9vBFmtnOmWZKZH8gO2tF26yU+phCAVaYuOoqATwOh3zeCOtg17jx+ZuIoTuNjWDlils8EXy2yMJ8ZZiWdgXBryqhvKzgDQ47/LV/YxYA7zm6CYF8rI3k44brd66eY06piYzDmikpH8lRhsw64Y7BtGs9MjM/FtrUV4gpKnXnmkO90YMI9mrntwpInJs7cnpYBdB0g2Q/Z3HNW6LbrNCDeMyp3PBMygbmuaB6k4vYF6iaCSIb3g5iE5u7STdXmwFpENE0uFtJth0vMs+270FMahS80lve2G3A4OstrvMd2H8G/uts+6tSErPoGrxe/FPyXnVyRLOkFz+Fk= 2022-03-10T16:27:36.583 INFO:teuthology.kill:2022-03-10 16:27:36,582.582 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-10T16:27:36.592 INFO:teuthology.kill:2022-03-10 16:27:36,592.592 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-10T16:27:36.662 INFO:teuthology.kill:2022-03-10 16:27:36,662.662 INFO:teuthology.orchestra.console:Power off smithi097 2022-03-10T16:27:36.681 INFO:teuthology.kill:2022-03-10 16:27:36,681.681 INFO:teuthology.orchestra.console:Power off smithi138 2022-03-10T16:27:45.048 INFO:teuthology.kill:2022-03-10 16:27:45,048.048 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2022-03-10T16:27:45.196 INFO:teuthology.kill:2022-03-10 16:27:45,195.195 INFO:teuthology.lock.ops:unlocked: smithi097.front.sepia.ceph.com 2022-03-10T16:27:49.169 INFO:teuthology.kill:2022-03-10 16:27:49,168.168 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-03-10T16:27:49.329 INFO:teuthology.kill:2022-03-10 16:27:49,329.329 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com 2022-03-10T16:29:49.682 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-10T16:29:49.816 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi097 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-10_15:28:20-rados-wip-yuri11-testing-2022-03-09-1706-octopus-distro-default-smithi/6730363 2022-03-10T16:29:49.817 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi138 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-10_15:28:20-rados-wip-yuri11-testing-2022-03-09-1706-octopus-distro-default-smithi/6730366