2022-03-09T23:41:38.003 INFO:root:teuthology version: 1.1.0-ff5db916 2022-03-09T23:41:38.127 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi018.front.sepia.ceph.com' reimaging 2022-03-09T23:41:38.128 INFO:teuthology.lock.ops:Updating [ubuntu@smithi018.front.sepia.ceph.com]: reset os type and version on server 2022-03-09T23:41:38.128 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2022-03-09T23:41:38.153 INFO:teuthology.lock.ops:Node 'ubuntu@smithi018.front.sepia.ceph.com' reimaging is complete 2022-03-09T23:41:38.153 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi161.front.sepia.ceph.com' reimaging 2022-03-09T23:41:38.154 INFO:teuthology.lock.ops:Updating [ubuntu@smithi161.front.sepia.ceph.com]: reset os type and version on server 2022-03-09T23:41:38.154 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2022-03-09T23:41:38.172 INFO:teuthology.lock.ops:Node 'ubuntu@smithi161.front.sepia.ceph.com' reimaging is complete 2022-03-09T23:41:38.384 INFO:teuthology.provision.fog.smithi018:Scheduling deploy of ubuntu 20.04 2022-03-09T23:41:38.392 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of ubuntu 20.04 2022-03-09T23:41:38.978 INFO:teuthology.orchestra.console:Power off smithi161 2022-03-09T23:41:39.155 INFO:teuthology.orchestra.console:Power off smithi018 2022-03-09T23:41:47.514 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2022-03-09T23:41:47.615 INFO:teuthology.orchestra.console:Power on smithi018 2022-03-09T23:41:51.453 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2022-03-09T23:41:51.554 INFO:teuthology.orchestra.console:Power on smithi161 2022-03-09T23:41:55.805 INFO:teuthology.orchestra.console:Power on for smithi161 completed 2022-03-09T23:41:55.906 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish 2022-03-09T23:41:56.089 INFO:teuthology.orchestra.console:Power on for smithi018 completed 2022-03-09T23:41:56.190 INFO:teuthology.provision.fog.smithi018:Waiting for deploy to finish 2022-03-09T23:46:43.604 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-09T23:46:44.101 INFO:teuthology.provision.fog.smithi018:Node is ready 2022-03-09T23:46:44.151 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018.front.sepia.ceph.com 2022-03-09T23:46:44.158 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18 smithi018.front.sepia.ceph.com smithi018 2022-03-09T23:46:44.340 INFO:teuthology.provision.fog.smithi018:Deploy complete! 2022-03-09T23:49:43.158 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-09T23:49:43.626 INFO:teuthology.provision.fog.smithi161:Node is ready 2022-03-09T23:49:43.633 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com 2022-03-09T23:49:43.640 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161 2022-03-09T23:49:43.900 INFO:teuthology.provision.fog.smithi161:Deploy complete! 2022-03-09T23:49:44.113 INFO:teuthology.lock.ops:Checking smithi018.front.sepia.ceph.com 2022-03-09T23:49:44.114 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-09T23:49:44.134 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com 2022-03-09T23:49:44.136 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-09T23:49:44.153 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: set os type and version on server 2022-03-09T23:49:44.154 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2022-03-09T23:49:44.461 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2022-03-09T23:49:44.467 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="Ubuntu" 2022-03-09T23:49:44.468 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-09T23:49:44.468 INFO:teuthology.orchestra.run.smithi018.stdout:ID=ubuntu 2022-03-09T23:49:44.468 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE=debian 2022-03-09T23:49:44.468 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-09T23:49:44.469 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="20.04" 2022-03-09T23:49:44.469 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-09T23:49:44.469 INFO:teuthology.orchestra.run.smithi018.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-09T23:49:44.469 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-09T23:49:44.469 INFO:teuthology.orchestra.run.smithi018.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-09T23:49:44.470 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_CODENAME=focal 2022-03-09T23:49:44.470 INFO:teuthology.orchestra.run.smithi018.stdout:UBUNTU_CODENAME=focal 2022-03-09T23:49:44.471 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2022-03-09T23:49:44.490 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server 2022-03-09T23:49:44.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2022-03-09T23:49:44.659 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2022-03-09T23:49:44.706 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Ubuntu" 2022-03-09T23:49:44.707 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-09T23:49:44.707 INFO:teuthology.orchestra.run.smithi161.stdout:ID=ubuntu 2022-03-09T23:49:44.708 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE=debian 2022-03-09T23:49:44.708 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-09T23:49:44.708 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="20.04" 2022-03-09T23:49:44.708 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-09T23:49:44.709 INFO:teuthology.orchestra.run.smithi161.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-09T23:49:44.709 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-09T23:49:44.709 INFO:teuthology.orchestra.run.smithi161.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-09T23:49:44.709 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_CODENAME=focal 2022-03-09T23:49:44.710 INFO:teuthology.orchestra.run.smithi161.stdout:UBUNTU_CODENAME=focal 2022-03-09T23:49:44.710 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2022-03-09T23:49:44.803 INFO:teuthology.dispatcher.supervisor:Running job 6728410 2022-03-09T23:49:44.804 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_ff5db9169566776b2f41faf671316ca58a318ea6/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-09_23:40:32-rgw-wip-yuri2-testing-2022-03-09-0859-pacific-distro-default-smithi/6728410 --name yuriw-2022-03-09_23:40:32-rgw-wip-yuri2-testing-2022-03-09-0859-pacific-distro-default-smithi --description rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} -- /home/teuthworker/archive/yuriw-2022-03-09_23:40:32-rgw-wip-yuri2-testing-2022-03-09-0859-pacific-distro-default-smithi/6728410/orig.config.yaml 2022-03-09T23:49:44.808 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-09_23:40:32-rgw-wip-yuri2-testing-2022-03-09-0859-pacific-distro-default-smithi/6728410 2022-03-09T23:49:44.809 INFO:teuthology.dispatcher.supervisor:Job PID: 5421 2022-03-09T23:49:44.809 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-10T06:20:13.561 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-10T06:20:13.882 INFO:teuthology.kill:Killing Pids: {5421} 2022-03-10T06:20:15.052 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-03-10T06:20:15.053 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-03-10T06:20:15.054 INFO:teuthology.misc:Compressing logs... 2022-03-10T06:20:15.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2022-03-10T06:20:15.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2022-03-10T06:20:15.591 INFO:teuthology.misc:Compressing logs... 2022-03-10T06:20:15.723 INFO:teuthology.misc:Compressing logs... 2022-03-10T06:22:14.563 INFO:teuthology.kill:No teuthology processes running 2022-03-10T06:22:15.795 INFO:teuthology.kill:Nuking machines: ['smithi018', 'smithi161'] 2022-03-10T06:22:16.775 INFO:teuthology.kill:2022-03-10 06:22:16,774.774 INFO:teuthology.nuke:targets: 2022-03-10T06:22:16.775 INFO:teuthology.kill: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDebbyXAClOrUgfng/tp1obwquhQAO9wZ6n04qhoFREIpkpdNdB8OmIcHElBkN0OipjRl9g4nZxnfXNCe5anzPVgJM2/qXMBik2Mdr3UPrg7LD7jmxoB7AK3nXi1dlL9El/eampB97cRiw+Nuv1z4dqavSX6M4C0pD4ITFoEgwGFxTIwKf4OXH9fcigCdaL2jNRfq8JKPbvE57THQFzW9QV5QBzyYREdYsjQWq6g3IorzqK9YPFVegGnz3Sq2/DVq/rqYWSNBB4/riRXt/H96ky3lYzlSO+HjKXmOiYyaRuvdtRV6qkDo/VESylqyzP6XkMrpkIqxMHIVXPIcJSyxdkvLRhaxSRAHBe3yC74mw21b03EjKxqbdABUGXRF/aXFccUzhaaNNLPKhYSESpFiARXhU73CuBnJHpePr0HSJHH47upomdwpABJEebc8po0CfZyxTRyemrMmQ/ROl27nHY+u8Yua/1h1aROsOWmNUYLBghoNNyOCCMdtyIojFMbQU= 2022-03-10T06:22:16.776 INFO:teuthology.kill: smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLa7bZygijexbKFDmNeIX731fTvlt2+9GuUbMs4tSfXhRN2osWlTj9SX0osDN+Lmo3B0CzkNhvncKvVGOjRtewjjlJ54uH3IPnKWrOg6d5CEyHv19E3fCzezvNlfrR1V4XLZs6zDbfQYxHQWLXKGjNrniHF4dFMj8Ojc7WNT0AMVUATWvHYeuY2uYYG5SQ/yLwc14zYoIaAMboGbq2X1BFjwnewzUcTDnWEvKt+KcIalaq8SxaeM/IsE+PTh58aCw5aAhFj4REaJf7ibyPWzWocLBo/bN2GYqFtW8MqjuYt7u2J1FaeQoROpPr7XRZ7FKlexYGeFSKCFbFWWra5IZqRIuTdjcKtrKhh+rX7jXYUM0ItUcfpS4xIjxZBN8S/8KFOfm8yTcUkSFwhbNISuB2iwujP8g2vCQYT0WxChsYSn2ClcDOihknKu5tPQydoZDQ6W3wdvBKch+QN6MlF16jBjoa2eoM6VtgpOPwF8EMJvuDy0N3GHkpSyZ8IiD6sSs= 2022-03-10T06:22:16.802 INFO:teuthology.kill:2022-03-10 06:22:16,802.802 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-10T06:22:16.807 INFO:teuthology.kill:2022-03-10 06:22:16,806.806 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-10T06:22:16.885 INFO:teuthology.kill:2022-03-10 06:22:16,885.885 INFO:teuthology.orchestra.console:Power off smithi161 2022-03-10T06:22:16.900 INFO:teuthology.kill:2022-03-10 06:22:16,899.899 INFO:teuthology.orchestra.console:Power off smithi018 2022-03-10T06:22:25.265 INFO:teuthology.kill:2022-03-10 06:22:25,265.265 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2022-03-10T06:22:25.416 INFO:teuthology.kill:2022-03-10 06:22:25,416.416 INFO:teuthology.lock.ops:unlocked: smithi018.front.sepia.ceph.com 2022-03-10T06:22:29.373 INFO:teuthology.kill:2022-03-10 06:22:29,373.373 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2022-03-10T06:22:29.520 INFO:teuthology.kill:2022-03-10 06:22:29,520.520 INFO:teuthology.lock.ops:unlocked: smithi161.front.sepia.ceph.com 2022-03-10T06:24:29.844 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-10T06:24:29.942 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi018 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-10_02:27:56-rados-wip-yuri-testing-2022-03-09-1540-distro-default-smithi/6729076 2022-03-10T06:24:29.942 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi161 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-10_02:27:56-rados-wip-yuri-testing-2022-03-09-1540-distro-default-smithi/6729075