2021-11-28T15:47:10.366 INFO:root:teuthology version: 1.1.0-a0bc3188 2021-11-28T15:47:10.482 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi087.front.sepia.ceph.com' reimaging 2021-11-28T15:47:10.483 INFO:teuthology.lock.ops:Updating [ubuntu@smithi087.front.sepia.ceph.com]: reset os type and version on server 2021-11-28T15:47:10.483 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2021-11-28T15:47:10.516 INFO:teuthology.lock.ops:Node 'ubuntu@smithi087.front.sepia.ceph.com' reimaging is complete 2021-11-28T15:47:10.516 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi102.front.sepia.ceph.com' reimaging 2021-11-28T15:47:10.517 INFO:teuthology.lock.ops:Updating [ubuntu@smithi102.front.sepia.ceph.com]: reset os type and version on server 2021-11-28T15:47:10.517 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2021-11-28T15:47:10.535 INFO:teuthology.lock.ops:Node 'ubuntu@smithi102.front.sepia.ceph.com' reimaging is complete 2021-11-28T15:47:11.047 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of ubuntu 20.04 2021-11-28T15:47:11.121 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of ubuntu 20.04 2021-11-28T15:47:14.132 INFO:teuthology.orchestra.console:Power off smithi102 2021-11-28T15:47:14.297 INFO:teuthology.orchestra.console:Power off smithi087 2021-11-28T15:47:26.619 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2021-11-28T15:47:26.721 INFO:teuthology.orchestra.console:Power on smithi102 2021-11-28T15:47:26.772 INFO:teuthology.orchestra.console:Power off for smithi087 completed 2021-11-28T15:47:26.873 INFO:teuthology.orchestra.console:Power on smithi087 2021-11-28T15:47:30.981 INFO:teuthology.orchestra.console:Power on for smithi102 completed 2021-11-28T15:47:31.083 INFO:teuthology.provision.fog.smithi102:Waiting for deploy to finish 2021-11-28T15:47:35.239 INFO:teuthology.orchestra.console:Power on for smithi087 completed 2021-11-28T15:47:35.340 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish 2021-11-28T15:52:43.787 ERROR:teuthology.orchestra.connection:Error authenticating with smithi102.front.sepia.ceph.com: Authentication failed. 2021-11-28T15:55:02.134 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-28T15:55:02.659 INFO:teuthology.provision.fog.smithi087:Node is ready 2021-11-28T15:55:02.665 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com 2021-11-28T15:55:02.673 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2021-11-28T15:55:02.840 INFO:teuthology.provision.fog.smithi087:Deploy complete! 2021-11-28T15:55:15.358 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-28T15:55:15.937 INFO:teuthology.provision.fog.smithi102:Node is ready 2021-11-28T15:55:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102.front.sepia.ceph.com 2021-11-28T15:55:15.993 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102 smithi102.front.sepia.ceph.com smithi102 2021-11-28T15:55:16.204 INFO:teuthology.provision.fog.smithi102:Deploy complete! 2021-11-28T15:55:16.381 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com 2021-11-28T15:55:16.382 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-28T15:55:16.401 INFO:teuthology.lock.ops:Checking smithi102.front.sepia.ceph.com 2021-11-28T15:55:16.402 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-28T15:55:16.423 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server 2021-11-28T15:55:16.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2021-11-28T15:55:16.594 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2021-11-28T15:55:16.601 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Ubuntu" 2021-11-28T15:55:16.601 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-11-28T15:55:16.601 INFO:teuthology.orchestra.run.smithi087.stdout:ID=ubuntu 2021-11-28T15:55:16.602 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE=debian 2021-11-28T15:55:16.602 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-11-28T15:55:16.602 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="20.04" 2021-11-28T15:55:16.602 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-28T15:55:16.603 INFO:teuthology.orchestra.run.smithi087.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-28T15:55:16.603 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-28T15:55:16.603 INFO:teuthology.orchestra.run.smithi087.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-28T15:55:16.603 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_CODENAME=focal 2021-11-28T15:55:16.604 INFO:teuthology.orchestra.run.smithi087.stdout:UBUNTU_CODENAME=focal 2021-11-28T15:55:16.605 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2021-11-28T15:55:16.624 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: set os type and version on server 2021-11-28T15:55:16.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2021-11-28T15:55:16.802 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2021-11-28T15:55:16.848 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="Ubuntu" 2021-11-28T15:55:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-11-28T15:55:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ID=ubuntu 2021-11-28T15:55:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE=debian 2021-11-28T15:55:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-11-28T15:55:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="20.04" 2021-11-28T15:55:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-28T15:55:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-28T15:55:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-28T15:55:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-28T15:55:16.851 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_CODENAME=focal 2021-11-28T15:55:16.851 INFO:teuthology.orchestra.run.smithi102.stdout:UBUNTU_CODENAME=focal 2021-11-28T15:55:16.851 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2021-11-28T15:55:16.952 INFO:teuthology.dispatcher.supervisor:Running job 6532028 2021-11-28T15:55:16.953 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a0bc318848afe71da1fbf34e1e1b44738d8e3d6d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-28_15:46:54-rgw-pacific-16.2.7_RC1-distro-default-smithi/6532028 --name yuriw-2021-11-28_15:46:54-rgw-pacific-16.2.7_RC1-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} -- /home/teuthworker/archive/yuriw-2021-11-28_15:46:54-rgw-pacific-16.2.7_RC1-distro-default-smithi/6532028/orig.config.yaml 2021-11-28T15:55:16.956 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-28_15:46:54-rgw-pacific-16.2.7_RC1-distro-default-smithi/6532028 2021-11-28T15:55:16.957 INFO:teuthology.dispatcher.supervisor:Job PID: 16212 2021-11-28T15:55:16.957 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-29T03:55:37.807 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-29T03:55:38.013 INFO:teuthology.kill:Killing Pids: {16212} 2021-11-29T03:55:39.163 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2021-11-29T03:55:39.163 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2021-11-29T03:55:39.164 INFO:teuthology.misc:Compressing logs... 2021-11-29T03:55:39.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2021-11-29T03:55:39.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2021-11-29T03:55:39.685 INFO:teuthology.misc:Compressing logs... 2021-11-29T03:55:39.811 INFO:teuthology.misc:Compressing logs... 2021-11-29T03:59:14.831 INFO:teuthology.kill:No teuthology processes running 2021-11-29T03:59:15.911 INFO:teuthology.kill:Nuking machines: ['smithi087', 'smithi102'] 2021-11-29T03:59:16.784 INFO:teuthology.kill:2021-11-29 03:59:16,784.784 INFO:teuthology.nuke:targets: 2021-11-29T03:59:16.785 INFO:teuthology.kill: smithi087.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+PRxTQca+ZvuzSYvepSqGd+QPEePqD6Q6e3wzNaMlJSm5EXauFlk6DlK/x6mLvT1Y3jSM6jmHxrjxTJF9pdZ3XCsG3zXIcfsOQW0vMbuDX68u3JJCLHp09WRLVfPeKulmx2N8ePiMDynTutUgs+wh15wiR6t/k0aOeoSOM9K6W5Ty0+4tQLxp+FG0CHav7xgRsJ2OyzVOoDdtLaZPaGmlj36JWTzqbWJDJUg6ktKXOx14vDLArY12aZ1ONS5PFCQtd0NlIj8dAn8Mvbipzpq94k2C7RVuzKXSSXMRMpuQ8Lqvw8yu6/a0bXIaa1R29tKLKTeT61Kxtkez2byBNI4MW+4yb6HEbDpKzARv7WvVBmo+ykdTGqAkFnYk1TH+ysphz18lUAK55OzIOf8YuccepoxFqQrzT2OeIBFadC+KWc/tFcMPPR566k1VtjXqFXi2szwlzO+Gk3pR0amZL+mmXpSl7ZgcNfHGnNpoPNBEwGEE3pxAjR5LzegGdXMgEo8= 2021-11-29T03:59:16.785 INFO:teuthology.kill: smithi102.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNbUzLY5lTusJXKy//LUszlVimEne9H7q4KKOp5YiyrnslyO2xrTizhLiWqy25ZATxS687ksznjKels9Su7c1K3Tls4UobLPjMY2eIoBuI75oS8kO6URig0KFOCPJcsGvJXs8md82LVnZUPu4/o/jEe1U6FxW9FCCT25xwW9r0V1Zh7Y9O3WQ22ldOubXw7YhbPGsZrCu9CzaDhiP7Qs66JVw3IRb2+ig2o6NS2MvHqNk/yro3Y+/p3Y1rtewvgc/BjeptUIUFJqYugVUnz9f0MEWTHurWedFZiMIH4ZYnaezpMRG+i8C5xISEKBLVGJxIAY6guB54AwCaLvOZdfmQf1BhzFr6w9H5e1IgsB8711OxtBv+XiclIc0hMM6aY7cGobO5QF6X3uUustcxz7olwkcIdgMQGxZsU5i9i6vsiLgBP5DBm1PEIBsFS0aQM15HpcnWl9w4yTYuAAcLv4sMZHOq7CF43pBFpAlTlnY9OicF4ZS8VXlr0UOLk+1Fc5E= 2021-11-29T03:59:16.809 INFO:teuthology.kill:2021-11-29 03:59:16,809.809 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-29T03:59:16.812 INFO:teuthology.kill:2021-11-29 03:59:16,812.812 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-29T03:59:16.889 INFO:teuthology.kill:2021-11-29 03:59:16,889.889 INFO:teuthology.orchestra.console:Power off smithi102 2021-11-29T03:59:16.902 INFO:teuthology.kill:2021-11-29 03:59:16,902.902 INFO:teuthology.orchestra.console:Power off smithi087 2021-11-29T03:59:25.259 INFO:teuthology.kill:2021-11-29 03:59:25,258.258 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2021-11-29T03:59:25.402 INFO:teuthology.kill:2021-11-29 03:59:25,401.401 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com 2021-11-29T03:59:29.378 INFO:teuthology.kill:2021-11-29 03:59:29,378.378 INFO:teuthology.orchestra.console:Power off for smithi087 completed 2021-11-29T03:59:29.523 INFO:teuthology.kill:2021-11-29 03:59:29,522.522 INFO:teuthology.lock.ops:unlocked: smithi087.front.sepia.ceph.com 2021-11-29T04:01:29.706 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15