2022-07-21T02:05:22.113 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5 2022-07-21T02:05:22.286 INFO:teuthology.lock.ops:Start node 'smithi145.front.sepia.ceph.com' reimaging 2022-07-21T02:05:22.292 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: reset os type and version on server 2022-07-21T02:05:22.292 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-07-21T02:05:22.321 INFO:teuthology.lock.ops:Node 'smithi145.front.sepia.ceph.com' reimaging is complete 2022-07-21T02:05:22.322 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging 2022-07-21T02:05:22.322 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server 2022-07-21T02:05:22.323 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2022-07-21T02:05:22.345 INFO:teuthology.lock.ops:Node 'smithi149.front.sepia.ceph.com' reimaging is complete 2022-07-21T02:05:22.556 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of ubuntu 20.04 2022-07-21T02:05:22.568 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of ubuntu 20.04 2022-07-21T02:05:22.986 INFO:teuthology.orchestra.console:Power off smithi149 2022-07-21T02:05:23.032 INFO:teuthology.orchestra.console:Power off smithi145 2022-07-21T02:05:23.143 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2022-07-21T02:05:23.176 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-07-21T02:05:23.245 INFO:teuthology.orchestra.console:Power on smithi149 2022-07-21T02:05:23.278 INFO:teuthology.orchestra.console:Power on smithi145 2022-07-21T02:05:31.618 INFO:teuthology.orchestra.console:Power on for smithi149 completed 2022-07-21T02:05:31.645 INFO:teuthology.orchestra.console:Power on for smithi145 completed 2022-07-21T02:05:31.720 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish 2022-07-21T02:05:31.746 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2022-07-21T02:10:24.630 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-21T02:10:25.096 INFO:teuthology.provision.fog.smithi145:Node is ready 2022-07-21T02:10:25.103 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2022-07-21T02:10:25.111 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2022-07-21T02:10:25.299 INFO:teuthology.provision.fog.smithi145:Deploy complete! 2022-07-21T02:10:32.294 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-21T02:10:32.807 INFO:teuthology.provision.fog.smithi149:Node is ready 2022-07-21T02:10:32.857 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com 2022-07-21T02:10:32.864 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149 2022-07-21T02:10:33.043 INFO:teuthology.provision.fog.smithi149:Deploy complete! 2022-07-21T02:10:33.269 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com 2022-07-21T02:10:33.270 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-21T02:10:33.292 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com 2022-07-21T02:10:33.292 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-21T02:10:33.313 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server 2022-07-21T02:10:33.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-07-21T02:10:33.567 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2022-07-21T02:10:33.583 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Ubuntu" 2022-07-21T02:10:33.583 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T02:10:33.584 INFO:teuthology.orchestra.run.smithi145.stdout:ID=ubuntu 2022-07-21T02:10:33.584 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE=debian 2022-07-21T02:10:33.584 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T02:10:33.584 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="20.04" 2022-07-21T02:10:33.585 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T02:10:33.585 INFO:teuthology.orchestra.run.smithi145.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T02:10:33.585 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T02:10:33.585 INFO:teuthology.orchestra.run.smithi145.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T02:10:33.585 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_CODENAME=focal 2022-07-21T02:10:33.586 INFO:teuthology.orchestra.run.smithi145.stdout:UBUNTU_CODENAME=focal 2022-07-21T02:10:33.587 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-07-21T02:10:33.619 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server 2022-07-21T02:10:33.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2022-07-21T02:10:33.897 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2022-07-21T02:10:33.955 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Ubuntu" 2022-07-21T02:10:33.957 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T02:10:33.958 INFO:teuthology.orchestra.run.smithi149.stdout:ID=ubuntu 2022-07-21T02:10:33.958 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE=debian 2022-07-21T02:10:33.959 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T02:10:33.959 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="20.04" 2022-07-21T02:10:33.960 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T02:10:33.961 INFO:teuthology.orchestra.run.smithi149.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T02:10:33.961 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T02:10:33.962 INFO:teuthology.orchestra.run.smithi149.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T02:10:33.962 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_CODENAME=focal 2022-07-21T02:10:33.964 INFO:teuthology.orchestra.run.smithi149.stdout:UBUNTU_CODENAME=focal 2022-07-21T02:10:33.965 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2022-07-21T02:10:34.094 INFO:teuthology.dispatcher.supervisor:Running job 6941403 2022-07-21T02:10:34.095 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2022-07-20_21:37:25-rgw-pacific-sec-distro-default-smithi/6941403 --name cbodley-2022-07-20_21:37:25-rgw-pacific-sec-distro-default-smithi --description rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} -- /home/teuthworker/archive/cbodley-2022-07-20_21:37:25-rgw-pacific-sec-distro-default-smithi/6941403/orig.config.yaml 2022-07-21T02:10:34.102 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2022-07-20_21:37:25-rgw-pacific-sec-distro-default-smithi/6941403 2022-07-21T02:10:34.103 INFO:teuthology.dispatcher.supervisor:Job PID: 9548 2022-07-21T02:10:34.104 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-21T14:11:37.794 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-21T14:11:38.023 INFO:teuthology.kill:Killing Pids: {9548} 2022-07-21T14:11:40.928 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-07-21T14:11:40.929 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-07-21T14:11:40.930 INFO:teuthology.misc:Compressing logs... 2022-07-21T14:11:40.931 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-07-21T14:11:41.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2022-07-21T14:11:41.741 INFO:teuthology.misc:Compressing logs... 2022-07-21T14:11:41.884 INFO:teuthology.misc:Compressing logs... 2022-07-21T14:11:53.011 INFO:teuthology.kill:No teuthology processes running 2022-07-21T14:11:56.539 INFO:teuthology.kill:Nuking machines: ['smithi145', 'smithi149'] 2022-07-21T14:12:00.299 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-21T14:12:00.301 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-21T14:12:00.301 INFO:teuthology.kill:2022-07-21 14:12:00,299.299 INFO:teuthology.nuke:targets: 2022-07-21T14:12:00.303 INFO:teuthology.kill: smithi145.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZFQyyr9Fd59zJM8qHj7gpkeQv6QIl+UoNNMOzWgk4IWuMATv9EVLznAgLGhM8NrwRUuZXw74zhnBg9bKcYUoKiIaxa9UD3Rz3hj1fmaJa8Nk1u3scqtfxX/hHO8KeATKIqF3ybFm3SL0SkGXcJhKLEUC12y6jZVDwhVYQUirayvLIPcBjoQgioGhLGbn9sVbQGWH1hDZqRkoQed9E8Q6N6PRu7qOzj4ivq4SFnn7a0f6QzRUa/VrX3JsRIjqUflmI/gxOnf/OEtO7dc67ZSgGMg+AyTEyBqjWDr3ZjADYaTpyGqFDnHWRDkey+KOv3P5ChYke5GcBb/nCSP04dTkGKKSgnQYzKbWtJvjKFxmDWU9+bMEg7iYFqnGEG3tOFdnHS3nvbuk1B718OYffALsJLSV0HxrbbTdpPwofpcX8f5UvWU4zgglRSI+HmVpdvbPAVq5IAXWeud/KJkLzBudPU8fth1UY52TrBlsu4LplDHX0sCZxnRbrtU2o8Dlvma8= 2022-07-21T14:12:00.303 INFO:teuthology.kill: smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuGJ/9VmvFc+T4o9sq4ES+R80t20PNh7XcOez6v5/itU2MF5DFv8fAWc+ttwvnnEVBCQJkQ57hn00FmjCPrE/zNdeAxk+7867rMdIsO7ElU2o34caUPSZu+dGiRGiPWQb6dMzIPXq4xfUcjYPFmPU+6khxpDspCLroGz1QaXirS/SZ8HraUyDFtcmapeQ0X0m+++kXgEKhC1rJ62mRXcXWyGfSzxS0so8JYMDwxXVHZkmtrpDzIg/t7wdJEKELm6iWgPWh799ODtL0SJrokXCcScqgdenN1+/BxZe7uDYFF29xbnfvZUA2CrMV729gJBcOC4bF6g4rxBkF51I4Ns4sZy5y8a+UeljJDSYYvBTuyr6iAPKpI5mhBcJC2Z32AEGx3R/8pZ0T9baKapc64Ef1oA4q/yTmJSWmeR/1v5ej9dsMZUMNHC13zMzkLx3lmx6q7tZkPWwhQpRGLExL5ZGIcum9FcjN/kgPq1Z4zaNzBkdB5VLdQ0cUcluQKE5QCV0= 2022-07-21T14:12:00.304 INFO:teuthology.kill:2022-07-21 14:12:00,304.304 INFO:teuthology.nuke:Checking targets against current locks 2022-07-21T14:12:00.442 INFO:teuthology.kill:2022-07-21 14:12:00,439.439 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-21T14:12:00.539 INFO:teuthology.kill:2022-07-21 14:12:00,538.538 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-21T14:12:00.696 INFO:teuthology.kill:2022-07-21 14:12:00,673.673 INFO:teuthology.orchestra.console:Power off smithi145 2022-07-21T14:12:00.843 INFO:teuthology.kill:2022-07-21 14:12:00,843.843 INFO:teuthology.orchestra.console:Power off smithi149 2022-07-21T14:12:09.379 INFO:teuthology.kill:2022-07-21 14:12:09,378.378 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2022-07-21T14:12:09.577 INFO:teuthology.kill:2022-07-21 14:12:09,575.575 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com 2022-07-21T14:12:13.482 INFO:teuthology.kill:2022-07-21 14:12:13,479.479 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-07-21T14:12:13.692 INFO:teuthology.kill:2022-07-21 14:12:13,692.692 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com 2022-07-21T14:14:14.932 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-21T14:14:15.431 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi145 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-07-21_13:43:33-rgw-quincy-sec-distro-default-smithi/6942487 2022-07-21T14:14:15.438 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi149 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-07-21_13:43:33-rgw-quincy-sec-distro-default-smithi/6942487