2021-11-23T06:59:58.345 INFO:root:teuthology version: 1.1.0-6cfddd5e
2021-11-23T06:59:58.471 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging
2021-11-23T06:59:58.472 INFO:teuthology.lock.ops:Updating [ubuntu@smithi017.front.sepia.ceph.com]: reset os type and version on server
2021-11-23T06:59:58.472 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server
2021-11-23T06:59:58.490 INFO:teuthology.lock.ops:Node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging is complete
2021-11-23T06:59:58.491 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi073.front.sepia.ceph.com' reimaging
2021-11-23T06:59:58.491 INFO:teuthology.lock.ops:Updating [ubuntu@smithi073.front.sepia.ceph.com]: reset os type and version on server
2021-11-23T06:59:58.491 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server
2021-11-23T06:59:58.510 INFO:teuthology.lock.ops:Node 'ubuntu@smithi073.front.sepia.ceph.com' reimaging is complete
2021-11-23T06:59:58.717 INFO:teuthology.provision.fog.smithi017:Scheduling deploy of ubuntu 20.04
2021-11-23T06:59:58.733 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of ubuntu 20.04
2021-11-23T06:59:59.134 INFO:teuthology.orchestra.console:Power off smithi017
2021-11-23T06:59:59.152 INFO:teuthology.orchestra.console:Power off smithi073
2021-11-23T07:00:07.561 INFO:teuthology.orchestra.console:Power off for smithi017 completed
2021-11-23T07:00:07.571 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2021-11-23T07:00:07.663 INFO:teuthology.orchestra.console:Power on smithi017
2021-11-23T07:00:07.674 INFO:teuthology.orchestra.console:Power on smithi073
2021-11-23T07:00:16.060 INFO:teuthology.orchestra.console:Power on for smithi017 completed
2021-11-23T07:00:16.097 INFO:teuthology.orchestra.console:Power on for smithi073 completed
2021-11-23T07:00:16.166 INFO:teuthology.provision.fog.smithi017:Waiting for deploy to finish
2021-11-23T07:00:16.204 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish
2021-11-23T07:03:17.936 ERROR:teuthology.orchestra.connection:Error authenticating with smithi073.front.sepia.ceph.com: Authentication failed.
2021-11-23T07:05:36.179 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-23T07:05:36.687 INFO:teuthology.provision.fog.smithi017:Node is ready
2021-11-23T07:05:36.694 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017.front.sepia.ceph.com
2021-11-23T07:05:36.702 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17 smithi017.front.sepia.ceph.com smithi017
2021-11-23T07:05:36.896 INFO:teuthology.provision.fog.smithi017:Deploy complete!
2021-11-23T07:06:02.716 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-23T07:06:03.272 INFO:teuthology.provision.fog.smithi073:Node is ready
2021-11-23T07:06:03.284 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073.front.sepia.ceph.com
2021-11-23T07:06:03.334 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73 smithi073.front.sepia.ceph.com smithi073
2021-11-23T07:06:03.584 INFO:teuthology.provision.fog.smithi073:Deploy complete!
2021-11-23T07:06:03.812 INFO:teuthology.lock.ops:Checking smithi017.front.sepia.ceph.com
2021-11-23T07:06:03.813 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-23T07:06:03.834 INFO:teuthology.lock.ops:Checking smithi073.front.sepia.ceph.com
2021-11-23T07:06:03.834 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-23T07:06:03.856 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: set os type and version on server
2021-11-23T07:06:03.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com'
2021-11-23T07:06:04.134 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64
2021-11-23T07:06:04.189 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Ubuntu"
2021-11-23T07:06:04.189 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-11-23T07:06:04.190 INFO:teuthology.orchestra.run.smithi017.stdout:ID=ubuntu
2021-11-23T07:06:04.190 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE=debian
2021-11-23T07:06:04.191 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-11-23T07:06:04.192 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="20.04"
2021-11-23T07:06:04.192 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.ubuntu.com/"
2021-11-23T07:06:04.192 INFO:teuthology.orchestra.run.smithi017.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-11-23T07:06:04.192 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-11-23T07:06:04.193 INFO:teuthology.orchestra.run.smithi017.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-11-23T07:06:04.193 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_CODENAME=focal
2021-11-23T07:06:04.194 INFO:teuthology.orchestra.run.smithi017.stdout:UBUNTU_CODENAME=focal
2021-11-23T07:06:04.197 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server
2021-11-23T07:06:04.218 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: set os type and version on server
2021-11-23T07:06:04.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com'
2021-11-23T07:06:04.408 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64
2021-11-23T07:06:04.457 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Ubuntu"
2021-11-23T07:06:04.458 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-11-23T07:06:04.458 INFO:teuthology.orchestra.run.smithi073.stdout:ID=ubuntu
2021-11-23T07:06:04.458 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE=debian
2021-11-23T07:06:04.458 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-11-23T07:06:04.459 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="20.04"
2021-11-23T07:06:04.459 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.ubuntu.com/"
2021-11-23T07:06:04.459 INFO:teuthology.orchestra.run.smithi073.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-11-23T07:06:04.459 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-11-23T07:06:04.460 INFO:teuthology.orchestra.run.smithi073.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-11-23T07:06:04.460 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_CODENAME=focal
2021-11-23T07:06:04.460 INFO:teuthology.orchestra.run.smithi073.stdout:UBUNTU_CODENAME=focal
2021-11-23T07:06:04.460 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server
2021-11-23T07:06:04.578 INFO:teuthology.dispatcher.supervisor:Running job 6521388
2021-11-23T07:06:04.578 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology -v --owner scheduled_prsrivas@teuthology --archive /home/teuthworker/archive/prsrivas-2021-11-23_05:27:16-rgw-wip-rgw-sts-correct-read-obj-perm-distro-basic-smithi/6521388 --name prsrivas-2021-11-23_05:27:16-rgw-wip-rgw-sts-correct-read-obj-perm-distro-basic-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{ubuntu_latest} tasks/test_multi} -- /home/teuthworker/archive/prsrivas-2021-11-23_05:27:16-rgw-wip-rgw-sts-correct-read-obj-perm-distro-basic-smithi/6521388/orig.config.yaml
2021-11-23T07:06:04.582 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/prsrivas-2021-11-23_05:27:16-rgw-wip-rgw-sts-correct-read-obj-perm-distro-basic-smithi/6521388
2021-11-23T07:06:04.583 INFO:teuthology.dispatcher.supervisor:Job PID: 21599
2021-11-23T07:06:04.583 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-23T19:06:26.209 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-23T19:06:26.402 INFO:teuthology.kill:Killing Pids: {21599}
2021-11-23T19:06:27.650 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1']
2021-11-23T19:06:27.651 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']
2021-11-23T19:06:27.651 INFO:teuthology.misc:Compressing logs...
2021-11-23T19:06:27.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com'
2021-11-23T19:06:27.843 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com'
2021-11-23T19:06:28.241 INFO:teuthology.misc:Compressing logs...
2021-11-23T19:06:28.382 INFO:teuthology.misc:Compressing logs...
2021-11-23T19:06:40.695 INFO:teuthology.kill:No teuthology processes running
2021-11-23T19:06:41.929 INFO:teuthology.kill:Nuking machines: ['smithi017', 'smithi073']
2021-11-23T19:06:42.980 INFO:teuthology.kill:2021-11-23 19:06:42,980.980 INFO:teuthology.nuke:targets:
2021-11-23T19:06:42.981 INFO:teuthology.kill:  smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeHrr9i7nZpjVt2KyXX/2O3TX7BMC5S+RJ0OxA6COBhWtpy76h1jKKCFCtMkYScFRINLwAtDOLDDls+boCtuWwIqxsz1o0kgKc8uxTzwqBmuC8T0BvS5GGbfEaRtFziv2vf9DQR3af1EwyqIvnfkH89bQzV+3siTZkPxrvcxu+8m+CU22bRgWT5V8qnqWKHtIhAo3sdumMyTTr9fXzR9EOq+DkC7v0jz4uftI2WZuyrzDR0tm+ucpgm3ozPYwakeX0dx433AgHccmnY0VLFszdnqvtCw5vY0V8pj3LLJjvSZYsenLWo1FnMwJ7laF5Ku5yCe1wUkvkdDDlUgvN9rSq5nwgf4jfsRQlkqjXrcn8OVipc2gU6lFrcrBuEVfjbkf3E4TSEpMDZqEZD+Lqm2QQs87yf3XLgp/l6YaqVJ1t6FqlbPx6azr0Wuhj+NYiL/MUihHHJOy4V0gb9pgpRmTMD87u2/Lt4jaZAx1RVoTWuxrWvUDfg5AVoUvPxIEXJQ8=
2021-11-23T19:06:42.981 INFO:teuthology.kill:  smithi073.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBYzrqUsvWMZ2f7Oas15oCf/RYZPhbD8bEJm+JpuRVi21JjIULomOj+t68tN2ExvVuRgIbW1GgrAxVFIlG7NzfihyGjvWl1eHbEtdygbgMzZ+bfwQK0kuR6tHt2nGK/Ft9q3NrHYw0fp50Uf8Q/tBs0jrlonJZSbbOwbqpDuTkKQgHx6cyAKmIBjcFm343OZxN4bJasTFpI9GIgZaJfDW9lXQhA/H2RbM+Hgpmrl16ouWM9hi+xDUSuEXxU6xeEUDr1RubAvh/YCS9/n9d0VNaXsLLlai6rQtQ4HFI7AnV6/2QoOsjhY32tJcrjK6/bd4kWylcfjq+lN3ZtZX0/N17s3Was+WZtp0ZqrQBMO0HsDvJolBr7DjeZMJuyzXzfxEF+tuod/CU0NJipsBeqZ+AYfx2vQBXjY6faMw+2V9rQO06BQwmtT32Y9MkF8lH9QvJgBtJtDFjpuCZbsMRKmYAfDCcYEPBaNC27kfoCW99t5mMigxiDx/TbWl63L6PRzU=
2021-11-23T19:06:43.008 INFO:teuthology.kill:2021-11-23 19:06:43,008.008 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-23T19:06:43.017 INFO:teuthology.kill:2021-11-23 19:06:43,017.017 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-23T19:06:43.086 INFO:teuthology.kill:2021-11-23 19:06:43,086.086 INFO:teuthology.orchestra.console:Power off smithi073
2021-11-23T19:06:43.101 INFO:teuthology.kill:2021-11-23 19:06:43,101.101 INFO:teuthology.orchestra.console:Power off smithi017
2021-11-23T19:06:51.467 INFO:teuthology.kill:2021-11-23 19:06:51,467.467 INFO:teuthology.orchestra.console:Power off for smithi017 completed
2021-11-23T19:06:51.618 INFO:teuthology.kill:2021-11-23 19:06:51,617.617 INFO:teuthology.lock.ops:unlocked: smithi017.front.sepia.ceph.com
2021-11-23T19:06:55.573 INFO:teuthology.kill:2021-11-23 19:06:55,572.572 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2021-11-23T19:06:55.717 INFO:teuthology.kill:2021-11-23 19:06:55,717.717 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com
2021-11-23T19:08:55.938 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-23T19:08:56.034 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi017 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522436
2021-11-23T19:08:56.035 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi073 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522439