2022-07-21T20:23:19.941 INFO:root:teuthology version: 1.1.1.dev389+g8d598431 2022-07-21T20:23:20.076 INFO:teuthology.lock.ops:Start node 'smithi101.front.sepia.ceph.com' reimaging 2022-07-21T20:23:20.077 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: reset os type and version on server 2022-07-21T20:23:20.078 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2022-07-21T20:23:20.098 INFO:teuthology.lock.ops:Node 'smithi101.front.sepia.ceph.com' reimaging is complete 2022-07-21T20:23:20.098 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging 2022-07-21T20:23:20.098 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server 2022-07-21T20:23:20.099 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2022-07-21T20:23:20.123 INFO:teuthology.lock.ops:Node 'smithi140.front.sepia.ceph.com' reimaging is complete 2022-07-21T20:23:20.491 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of ubuntu 20.04 2022-07-21T20:23:20.497 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of ubuntu 20.04 2022-07-21T20:23:21.713 INFO:teuthology.orchestra.console:Power off smithi140 2022-07-21T20:23:21.765 INFO:teuthology.orchestra.console:Power off smithi101 2022-07-21T20:23:21.858 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2022-07-21T20:23:21.903 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2022-07-21T20:23:21.960 INFO:teuthology.orchestra.console:Power on smithi140 2022-07-21T20:23:22.004 INFO:teuthology.orchestra.console:Power on smithi101 2022-07-21T20:23:26.252 INFO:teuthology.orchestra.console:Power on for smithi101 completed 2022-07-21T20:23:26.353 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish 2022-07-21T20:23:30.329 INFO:teuthology.orchestra.console:Power on for smithi140 completed 2022-07-21T20:23:30.430 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2022-07-21T20:28:35.524 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-21T20:28:36.045 INFO:teuthology.provision.fog.smithi101:Node is ready 2022-07-21T20:28:36.094 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com 2022-07-21T20:28:36.132 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101 2022-07-21T20:28:36.488 INFO:teuthology.provision.fog.smithi101:Deploy complete! 2022-07-21T20:28:51.526 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-21T20:28:52.061 INFO:teuthology.provision.fog.smithi140:Node is ready 2022-07-21T20:28:52.108 INFO:teuthology.orchestra.run.smithi140.stdout:smithi140.front.sepia.ceph.com 2022-07-21T20:28:52.178 INFO:teuthology.orchestra.run.smithi140.stdout:172.21.15.140 smithi140.front.sepia.ceph.com smithi140 2022-07-21T20:28:52.609 INFO:teuthology.provision.fog.smithi140:Deploy complete! 2022-07-21T20:28:53.136 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com 2022-07-21T20:28:53.137 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-21T20:28:53.203 INFO:teuthology.lock.ops:Checking smithi140.front.sepia.ceph.com 2022-07-21T20:28:53.204 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-21T20:28:53.283 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server 2022-07-21T20:28:53.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2022-07-21T20:28:53.861 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2022-07-21T20:28:53.909 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Ubuntu" 2022-07-21T20:28:53.909 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T20:28:53.910 INFO:teuthology.orchestra.run.smithi101.stdout:ID=ubuntu 2022-07-21T20:28:53.910 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE=debian 2022-07-21T20:28:53.910 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T20:28:53.911 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="20.04" 2022-07-21T20:28:53.911 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T20:28:53.911 INFO:teuthology.orchestra.run.smithi101.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T20:28:53.911 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T20:28:53.912 INFO:teuthology.orchestra.run.smithi101.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T20:28:53.912 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_CODENAME=focal 2022-07-21T20:28:53.913 INFO:teuthology.orchestra.run.smithi101.stdout:UBUNTU_CODENAME=focal 2022-07-21T20:28:53.915 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2022-07-21T20:28:53.962 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: set os type and version on server 2022-07-21T20:28:53.963 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi140.front.sepia.ceph.com' 2022-07-21T20:28:54.488 INFO:teuthology.orchestra.run.smithi140.stdout:x86_64 2022-07-21T20:28:54.503 INFO:teuthology.orchestra.run.smithi140.stdout:NAME="Ubuntu" 2022-07-21T20:28:54.503 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T20:28:54.504 INFO:teuthology.orchestra.run.smithi140.stdout:ID=ubuntu 2022-07-21T20:28:54.504 INFO:teuthology.orchestra.run.smithi140.stdout:ID_LIKE=debian 2022-07-21T20:28:54.504 INFO:teuthology.orchestra.run.smithi140.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T20:28:54.505 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_ID="20.04" 2022-07-21T20:28:54.505 INFO:teuthology.orchestra.run.smithi140.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T20:28:54.535 INFO:teuthology.orchestra.run.smithi140.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T20:28:54.535 INFO:teuthology.orchestra.run.smithi140.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T20:28:54.536 INFO:teuthology.orchestra.run.smithi140.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T20:28:54.537 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_CODENAME=focal 2022-07-21T20:28:54.538 INFO:teuthology.orchestra.run.smithi140.stdout:UBUNTU_CODENAME=focal 2022-07-21T20:28:54.549 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2022-07-21T20:28:54.811 INFO:teuthology.dispatcher.supervisor:Running job 6943091 2022-07-21T20:28:54.812 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_8d598431210977f8caccec83230b4bfec7bd5d3f/virtualenv/bin/teuthology -v --owner scheduled_dgalloway@teuthology --archive /home/teuthworker/archive/dgalloway-2022-07-21_20:18:50-rgw-pacific-distro-default-smithi/6943091 --name dgalloway-2022-07-21_20:18:50-rgw-pacific-distro-default-smithi --description rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} -- /home/teuthworker/archive/dgalloway-2022-07-21_20:18:50-rgw-pacific-distro-default-smithi/6943091/orig.config.yaml 2022-07-21T20:28:54.822 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dgalloway-2022-07-21_20:18:50-rgw-pacific-distro-default-smithi/6943091 2022-07-21T20:28:54.823 INFO:teuthology.dispatcher.supervisor:Job PID: 21592 2022-07-21T20:28:54.824 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-22T08:29:26.619 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-22T08:29:26.805 INFO:teuthology.kill:Killing Pids: {21592} 2022-07-22T08:29:28.312 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-07-22T08:29:28.313 INFO:teuthology.task.internal:roles: ubuntu@smithi140.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-07-22T08:29:28.313 INFO:teuthology.misc:Compressing logs... 2022-07-22T08:29:28.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2022-07-22T08:29:28.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi140.front.sepia.ceph.com' 2022-07-22T08:29:28.957 INFO:teuthology.misc:Compressing logs... 2022-07-22T08:29:29.099 INFO:teuthology.misc:Compressing logs... 2022-07-22T08:33:19.303 INFO:teuthology.kill:No teuthology processes running 2022-07-22T08:33:20.701 INFO:teuthology.kill:Nuking machines: ['smithi101', 'smithi140'] 2022-07-22T08:33:21.841 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_8d598431210977f8caccec83230b4bfec7bd5d3f/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-22T08:33:21.842 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-22T08:33:21.842 INFO:teuthology.kill:2022-07-22 08:33:21,840.840 INFO:teuthology.nuke:targets: 2022-07-22T08:33:21.842 INFO:teuthology.kill: smithi101.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCeChoYrOKQVtO9Bvv9pW27yQeRyLRN0FCDnESoQamAz1OfnCeytw0jHFKf1fLPlf3Um4WBH3vyJOkOAaAHBwjGHZSzLqVJB1d6xYzn4BqZWe704nWfnCCR/5GWMHcN0h7sZBsOW5ndw6Kp8eXtkapPL7LCnaKop8ObPsqx0+eOL9rz/NI7qz1XCZ+tVSVEBmB7tshy+IfyUFQkxLEPafF87iqLh0S76ogdO14xOchYZhQgLloSbnd2uMlIWCzpi3TrXZ2QoUSBc0x3dtSFzM8LOvLMs7UD4cAyKVZJIrfj0CqtkgK15dGVzMqkuF7rAH4SDKXdCSAaify3xedzCjYoS+5DLdLUsCLL22Z/jUHkswuekb7gZk7dO4cCbt/IQj4VhNMdXd+RXqsEj5RMs5JpJAxuvLf5i9fsdF8toHv+oB6xKOk3Iq8sLmQ7xSzFBTXxyha+KdFHI5clFViuBlP2wXX0nqa1T1xeVV+u70LhIoIlaBWjt+eksnFk4IYUT50= 2022-07-22T08:33:21.843 INFO:teuthology.kill: smithi140.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOFXI7OIyAlYPYSEOFwiiawIPeX9JmWUCEw13soGz7zag0hZMC0bDPPoUEjEBDLTRo9Ptdcp5Lv4dhZTNw2M3RAbmWZhUj1lTguDg8AHeftzGKUBB6xo5fjYHIOv4PXtZZ9m9c1+1pAzBlcAjv58nBwQs9bUFmbDByfUAJdLDdXNpGimZuLsmm597o9SbghoX2BKtPKVtIbOIW7OyU+cfy9+kAaTXkjwC+le8UxuIDTVdu8/sSNB2JxYVKyImboReJF0wyD1LMl7etbCgIxonJojxZuz+bJ2GDbhiDhQ0NVtV5uKoIxKBatHnsa0FCXvN2OXSMSaQgsoyARhSsesDNlqXFb+CDhnhOu1eQltl0uX+ubMsu9Hbesh4zlCYWzIzPM0bmt14XDicUB5gdiuGBsxRo5OEwzYz6B7fmCSNMKccpWpLjzoFPJWLYVR+cbVE2gNy0vDW/LsaHwrzGsFR6pmD5vYzm8MpWZOH40OXXBDtttoXHO3QZHhCwEBSjUzU= 2022-07-22T08:33:21.843 INFO:teuthology.kill:2022-07-22 08:33:21,841.841 INFO:teuthology.nuke:Checking targets against current locks 2022-07-22T08:33:21.902 INFO:teuthology.kill:2022-07-22 08:33:21,902.902 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-22T08:33:21.935 INFO:teuthology.kill:2022-07-22 08:33:21,934.934 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-22T08:33:21.998 INFO:teuthology.kill:2022-07-22 08:33:21,998.998 INFO:teuthology.orchestra.console:Power off smithi101 2022-07-22T08:33:22.045 INFO:teuthology.kill:2022-07-22 08:33:22,045.045 INFO:teuthology.orchestra.console:Power off smithi140 2022-07-22T08:33:30.412 INFO:teuthology.kill:2022-07-22 08:33:30,412.412 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2022-07-22T08:33:30.567 INFO:teuthology.kill:2022-07-22 08:33:30,566.566 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2022-07-22T08:33:34.493 INFO:teuthology.kill:2022-07-22 08:33:34,493.493 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2022-07-22T08:33:34.648 INFO:teuthology.kill:2022-07-22 08:33:34,648.648 INFO:teuthology.lock.ops:unlocked: smithi101.front.sepia.ceph.com 2022-07-22T08:35:34.985 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-22T08:35:35.099 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi101 but it was locked by another job: /home/teuthworker/archive/pdonnell-2022-07-22_03:35:24-fs-wip-pdonnell-testing-20220721.235756-distro-default-smithi/6943988 2022-07-22T08:35:35.100 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi140 but it was locked by another job: /home/teuthworker/archive/pdonnell-2022-07-22_03:35:24-fs-wip-pdonnell-testing-20220721.235756-distro-default-smithi/6943988