2022-02-24T16:30:00.124 INFO:root:teuthology version: 1.1.0-6b607b46 2022-02-24T16:30:00.249 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi100.front.sepia.ceph.com' reimaging 2022-02-24T16:30:00.250 INFO:teuthology.lock.ops:Updating [ubuntu@smithi100.front.sepia.ceph.com]: reset os type and version on server 2022-02-24T16:30:00.250 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2022-02-24T16:30:00.271 INFO:teuthology.lock.ops:Node 'ubuntu@smithi100.front.sepia.ceph.com' reimaging is complete 2022-02-24T16:30:00.271 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi102.front.sepia.ceph.com' reimaging 2022-02-24T16:30:00.272 INFO:teuthology.lock.ops:Updating [ubuntu@smithi102.front.sepia.ceph.com]: reset os type and version on server 2022-02-24T16:30:00.272 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2022-02-24T16:30:00.291 INFO:teuthology.lock.ops:Node 'ubuntu@smithi102.front.sepia.ceph.com' reimaging is complete 2022-02-24T16:30:00.494 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of ubuntu 20.04 2022-02-24T16:30:00.510 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of ubuntu 20.04 2022-02-24T16:30:00.817 INFO:teuthology.orchestra.console:Power off smithi100 2022-02-24T16:30:00.833 INFO:teuthology.orchestra.console:Power off smithi102 2022-02-24T16:30:09.188 INFO:teuthology.orchestra.console:Power off for smithi100 completed 2022-02-24T16:30:09.290 INFO:teuthology.orchestra.console:Power on smithi100 2022-02-24T16:30:13.303 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2022-02-24T16:30:13.405 INFO:teuthology.orchestra.console:Power on smithi102 2022-02-24T16:30:17.649 INFO:teuthology.orchestra.console:Power on for smithi100 completed 2022-02-24T16:30:17.654 INFO:teuthology.orchestra.console:Power on for smithi102 completed 2022-02-24T16:30:17.751 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish 2022-02-24T16:30:17.755 INFO:teuthology.provision.fog.smithi102:Waiting for deploy to finish 2022-02-24T16:35:12.391 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-24T16:35:12.872 INFO:teuthology.provision.fog.smithi102:Node is ready 2022-02-24T16:35:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102.front.sepia.ceph.com 2022-02-24T16:35:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102 smithi102.front.sepia.ceph.com smithi102 2022-02-24T16:35:13.131 INFO:teuthology.provision.fog.smithi102:Deploy complete! 2022-02-24T16:35:14.278 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-24T16:35:14.736 INFO:teuthology.provision.fog.smithi100:Node is ready 2022-02-24T16:35:14.743 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com 2022-02-24T16:35:14.792 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100 2022-02-24T16:35:14.961 INFO:teuthology.provision.fog.smithi100:Deploy complete! 2022-02-24T16:35:15.183 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com 2022-02-24T16:35:15.184 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-24T16:35:15.203 INFO:teuthology.lock.ops:Checking smithi102.front.sepia.ceph.com 2022-02-24T16:35:15.204 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-24T16:35:15.223 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server 2022-02-24T16:35:15.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2022-02-24T16:35:15.402 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2022-02-24T16:35:15.454 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu" 2022-02-24T16:35:15.454 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-24T16:35:15.455 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu 2022-02-24T16:35:15.455 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian 2022-02-24T16:35:15.455 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-24T16:35:15.456 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="20.04" 2022-02-24T16:35:15.456 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-24T16:35:15.456 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-24T16:35:15.457 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-24T16:35:15.457 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-24T16:35:15.457 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=focal 2022-02-24T16:35:15.457 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=focal 2022-02-24T16:35:15.459 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2022-02-24T16:35:15.478 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: set os type and version on server 2022-02-24T16:35:15.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2022-02-24T16:35:15.650 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2022-02-24T16:35:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="Ubuntu" 2022-02-24T16:35:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-24T16:35:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:ID=ubuntu 2022-02-24T16:35:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE=debian 2022-02-24T16:35:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-24T16:35:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="20.04" 2022-02-24T16:35:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-24T16:35:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-24T16:35:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-24T16:35:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-24T16:35:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_CODENAME=focal 2022-02-24T16:35:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:UBUNTU_CODENAME=focal 2022-02-24T16:35:15.660 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2022-02-24T16:35:15.757 INFO:teuthology.dispatcher.supervisor:Running job 6704582 2022-02-24T16:35:15.757 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6b607b46862a6b148fb1608e09b31d367f094555/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-02-24_15:41:20-rgw-wip-yuri2-testing-2022-02-23-1524-pacific-distro-default-smithi/6704582 --name yuriw-2022-02-24_15:41:20-rgw-wip-yuri2-testing-2022-02-23-1524-pacific-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} -- /home/teuthworker/archive/yuriw-2022-02-24_15:41:20-rgw-wip-yuri2-testing-2022-02-23-1524-pacific-distro-default-smithi/6704582/orig.config.yaml 2022-02-24T16:35:15.760 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-02-24_15:41:20-rgw-wip-yuri2-testing-2022-02-23-1524-pacific-distro-default-smithi/6704582 2022-02-24T16:35:15.762 INFO:teuthology.dispatcher.supervisor:Job PID: 17877 2022-02-24T16:35:15.762 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-24T23:05:44.451 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-24T23:05:44.645 INFO:teuthology.kill:Killing Pids: {17877} 2022-02-24T23:05:45.770 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-02-24T23:05:45.771 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-02-24T23:05:45.772 INFO:teuthology.misc:Compressing logs... 2022-02-24T23:05:45.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2022-02-24T23:05:45.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2022-02-24T23:05:46.283 INFO:teuthology.misc:Compressing logs... 2022-02-24T23:05:46.413 INFO:teuthology.misc:Compressing logs... 2022-02-24T23:07:46.161 INFO:teuthology.kill:No teuthology processes running 2022-02-24T23:07:47.285 INFO:teuthology.kill:Nuking machines: ['smithi100', 'smithi102'] 2022-02-24T23:07:48.259 INFO:teuthology.kill:2022-02-24 23:07:48,258.258 INFO:teuthology.nuke:targets: 2022-02-24T23:07:48.260 INFO:teuthology.kill: smithi100.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4JREkVc44oX3ODEN/CLxWK7HIMJAOuvCD9BG3fnR8AiuP0dp3UJ1p1yA6Di2/wRhP8iH5ZjSD7br9jwa92+xDSqpj8umj2QTedtXglY6hrEp7LEgy2t5QYh5mTTPKn1m5I9BBSlDwzz26FajVZrG8nVQvhX7gyjMu1WCQ6Y5Eblz4hvpMyX3Wn2o+L4boZy5lRHnOyceVVkdPxVhN35tPbUhp2v4+XwoZCy1SwWdBrnxxzJ8JrrGlmwj29b+9onU0K9E1uvkXayLwVfuT+tK7ZtEOE/MU8nctBllUUNiTQ/OJzdzUD6aO2MGtX9F6wP1HT89Smr6KPLNnz7jc0AogwZIqcW9XiC7+VBXyfsejuywRJxfYFmJyfV/eD1cstqDEGpGP3c81cXO+kfaMuRqJvs23bDS06LMf27KR/so+2SECYFZx3xSLh3crJT95XxFjES95mxKTQgDBnSWftJi/cvSFUa8Neh20jzJ36Y67NKMlLGYxXSIEOl4bMiDeTTs= 2022-02-24T23:07:48.260 INFO:teuthology.kill: smithi102.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4+58lvSkaULDcbqbSVuzbMVbuukqRgV+h77UWqjcBg07qwbDkrItcL+w52mMlcdpRLxRt7QUbfupZ8kyAu6xiTCSwgKoCw00JJB/bgb0mGHB/72COBACah21E4eag+D9mq/8lUIIVcV8FF1VKSibc0JPuOhPZbjUh6vGA4Y3y3YHE5PNnH3hjgQmw9gImCqsNt6i/qzkullqaZo9sic4tNModa3s/JpLAAIXUUNltgxRLclMspdBXbDWzBWd1OiThiGNsqaYgwTVGUje9i6AA2AwWc+/GOYfY2pr+Cz82GXvYfAVKGoGMN9T29kCQpuHrW/okLbblm4Ws6oOsHVwFJsUMS5YWUIMvPpfyxphMEaer3fJfTq5+le95YjWhPZL5VBBR7kfj5g7d05Ld0KIozmuBsZWg+BjWqPziUg4s4YVA/rtB65g2yjKv9H8IknIfqcfNyo9lNe2wxzTYNJOQnnb7CwH4Q+rNBg73yscpPhF0R70v1i6xfaZpDmYHLOM= 2022-02-24T23:07:48.285 INFO:teuthology.kill:2022-02-24 23:07:48,284.284 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-24T23:07:48.288 INFO:teuthology.kill:2022-02-24 23:07:48,288.288 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-24T23:07:48.363 INFO:teuthology.kill:2022-02-24 23:07:48,363.363 INFO:teuthology.orchestra.console:Power off smithi102 2022-02-24T23:07:48.377 INFO:teuthology.kill:2022-02-24 23:07:48,377.377 INFO:teuthology.orchestra.console:Power off smithi100 2022-02-24T23:07:56.747 INFO:teuthology.kill:2022-02-24 23:07:56,746.746 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2022-02-24T23:07:56.747 INFO:teuthology.kill:2022-02-24 23:07:56,747.747 INFO:teuthology.orchestra.console:Power off for smithi100 completed 2022-02-24T23:07:56.897 INFO:teuthology.kill:2022-02-24 23:07:56,897.897 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com 2022-02-24T23:07:56.906 INFO:teuthology.kill:2022-02-24 23:07:56,905.905 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com 2022-02-24T23:09:57.226 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-24T23:09:57.325 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi100 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-24_22:32:08-fs-wip-yuri3-testing-2022-02-22-0946-pacific-distro-default-smithi/6704813 2022-02-24T23:09:57.326 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi102 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-24_22:32:08-fs-wip-yuri3-testing-2022-02-22-0946-pacific-distro-default-smithi/6704813