2022-09-03T05:32:09.952 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-03T05:32:10.153 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging 2022-09-03T05:32:10.155 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server 2022-09-03T05:32:10.155 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2022-09-03T05:32:10.188 INFO:teuthology.lock.ops:Node 'smithi062.front.sepia.ceph.com' reimaging is complete 2022-09-03T05:32:10.189 INFO:teuthology.lock.ops:Start node 'smithi087.front.sepia.ceph.com' reimaging 2022-09-03T05:32:10.190 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: reset os type and version on server 2022-09-03T05:32:10.190 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2022-09-03T05:32:10.218 INFO:teuthology.lock.ops:Node 'smithi087.front.sepia.ceph.com' reimaging is complete 2022-09-03T05:32:10.421 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of centos 8.stream 2022-09-03T05:32:10.443 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of centos 8.stream 2022-09-03T05:32:10.792 INFO:teuthology.orchestra.console:Power off smithi062 2022-09-03T05:32:10.853 INFO:teuthology.orchestra.console:Power off smithi087 2022-09-03T05:32:23.311 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2022-09-03T05:32:23.358 INFO:teuthology.orchestra.console:Power off for smithi087 completed 2022-09-03T05:32:23.413 INFO:teuthology.orchestra.console:Power on smithi062 2022-09-03T05:32:23.460 INFO:teuthology.orchestra.console:Power on smithi087 2022-09-03T05:32:27.670 INFO:teuthology.orchestra.console:Power on for smithi062 completed 2022-09-03T05:32:27.717 INFO:teuthology.orchestra.console:Power on for smithi087 completed 2022-09-03T05:32:27.773 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish 2022-09-03T05:32:27.820 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish 2022-09-03T05:37:05.907 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-03T05:37:06.199 INFO:teuthology.provision.fog.smithi087:Node is ready 2022-09-03T05:37:06.219 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com 2022-09-03T05:37:06.279 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2022-09-03T05:37:06.631 INFO:teuthology.provision.fog.smithi087:Deploy complete! 2022-09-03T05:37:14.942 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-03T05:37:15.246 INFO:teuthology.provision.fog.smithi062:Node is ready 2022-09-03T05:37:15.269 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com 2022-09-03T05:37:15.329 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062 2022-09-03T05:37:15.712 INFO:teuthology.provision.fog.smithi062:Deploy complete! 2022-09-03T05:37:15.926 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com 2022-09-03T05:37:15.928 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-03T05:37:15.957 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com 2022-09-03T05:37:15.957 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-03T05:37:15.987 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server 2022-09-03T05:37:15.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2022-09-03T05:37:16.248 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2022-09-03T05:37:16.307 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2022-09-03T05:37:16.307 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8" 2022-09-03T05:37:16.308 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2022-09-03T05:37:16.308 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2022-09-03T05:37:16.308 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8" 2022-09-03T05:37:16.308 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2022-09-03T05:37:16.309 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-03T05:37:16.309 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2022-09-03T05:37:16.309 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-03T05:37:16.309 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2022-09-03T05:37:16.310 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-03T05:37:16.310 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-03T05:37:16.310 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-03T05:37:16.312 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2022-09-03T05:37:16.334 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server 2022-09-03T05:37:16.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2022-09-03T05:37:16.551 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2022-09-03T05:37:16.611 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2022-09-03T05:37:16.614 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8" 2022-09-03T05:37:16.614 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2022-09-03T05:37:16.615 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2022-09-03T05:37:16.615 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8" 2022-09-03T05:37:16.615 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2022-09-03T05:37:16.615 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-03T05:37:16.616 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2022-09-03T05:37:16.616 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-03T05:37:16.616 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2022-09-03T05:37:16.616 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-03T05:37:16.616 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-03T05:37:16.617 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-03T05:37:16.617 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2022-09-03T05:37:16.721 INFO:teuthology.dispatcher.supervisor:Running job 7008075 2022-09-03T05:37:16.722 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2022-09-02_14:33:28-rgw-pr-47848-distro-default-smithi/7008075 --name cbodley-2022-09-02_14:33:28-rgw-pr-47848-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi} -- /home/teuthworker/archive/cbodley-2022-09-02_14:33:28-rgw-pr-47848-distro-default-smithi/7008075/orig.config.yaml 2022-09-03T05:37:16.727 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2022-09-02_14:33:28-rgw-pr-47848-distro-default-smithi/7008075 2022-09-03T05:37:16.728 INFO:teuthology.dispatcher.supervisor:Job PID: 5892 2022-09-03T05:37:16.728 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-03T17:38:16.709 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-03T17:38:16.880 INFO:teuthology.kill:Killing Pids: {5892} 2022-09-03T17:38:18.369 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-09-03T17:38:18.370 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-09-03T17:38:18.370 INFO:teuthology.misc:Compressing logs... 2022-09-03T17:38:18.371 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2022-09-03T17:38:18.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2022-09-03T17:38:19.135 INFO:teuthology.misc:Compressing logs... 2022-09-03T17:38:36.387 INFO:teuthology.misc:Compressing logs... 2022-09-03T17:59:18.128 INFO:teuthology.kill:No teuthology processes running 2022-09-03T17:59:19.566 INFO:teuthology.kill:Nuking machines: ['smithi062', 'smithi087'] 2022-09-03T17:59:20.705 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-03T17:59:20.706 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-03T17:59:20.706 INFO:teuthology.kill:2022-09-03 17:59:20,704.704 INFO:teuthology.nuke:targets: 2022-09-03T17:59:20.706 INFO:teuthology.kill: smithi062.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSHRk+FEtEk0j2RQdYKdyMR/5SvhQ+BPYn0gwmkL5EAzXE5undqYEXihmsS0eF0BPoaxbe6Iw7OEnb3csdEOS1E5PFlWUcn1eJII8vdIiboGKEUVStcXyfSIi5xvHeLzDMmy0Oe2yj4J+mlHSpkntH9oDX8tjTwNf4wPzVI2hk9WWyB1ZDrdWZxtO5QjkecSTp10ZPZS6tbWJtxAFSEiZl5lMo/gi6fnRnD4uHf2xHMQa87pUknbtbobXHed/NFAKM+a/iBBNy7kdtQsFOjpbK02N+cHuYdAOrd4lTuZK3LPuTnBuw9CXESr03blNlAPK8nz/0mdmAysyIidk00OUflY0oN0aY9qgiyQ/f8jElda8reLmrqq1kFG9P7p1roLg+CBgSS2ytsV8nyLHhA0HJHuGdz9+7hOgfdpWsN2wBRez34hFLzoYhCzdd9HycuDp9dK5A3VaTzQ9owbXSfF/1MIFnqGpeUrtUl3//kuN5JBR03Tu31Ahq93aoEzcRFkE= 2022-09-03T17:59:20.707 INFO:teuthology.kill: smithi087.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA2MfLiBfBB1rQjxhGdqrEUq/1La6/wNGaBQ7PWfp8JYfQ54QThAn+gKdHG9f6dimygPtacq48uajMWnuqetlFwFvkL0kiK0e6dWMTVvD58jmrP8ymFkir0uO+4/i+CGGxFAEbuqtCah38yHkyefQ08/UBmZBsQqcN87vAJIBLiF3JjPTrF5nx8L4s09mXEcOMA7TkUxQhCOcIOMzmFF0NNx0dIT9+FGCmHCxnoI3xj4s+o94e2OFcdwXHBmBqGKoNHLpcLsVpIbScZyLkVmLamfLkrmIC7I5bKZO/OxIk7Yhv/QLRGf6+0wdnh0kQ6ZlRmhXbk9hqQ7rv+g6P+0GyAkIgpDw99T5eoGm8GiGB7ERNuwHiUS5zV+HZ0XcxC8AKa0uH58JrSJmM2PvQLnVhjLMjwPWNXtTymDoaSRHDf+cMK0XfQc1QOBvl26wZ/KIYXD+3H6ekFX62KG2d9hPZ2QQf2JWJZUDCzL3A9+W9ortahXotXOx4rzG+iyxI+DU= 2022-09-03T17:59:20.707 INFO:teuthology.kill:2022-09-03 17:59:20,705.705 INFO:teuthology.nuke:Checking targets against current locks 2022-09-03T17:59:20.764 INFO:teuthology.kill:2022-09-03 17:59:20,763.763 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-03T17:59:20.790 INFO:teuthology.kill:2022-09-03 17:59:20,789.789 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-03T17:59:20.851 INFO:teuthology.kill:2022-09-03 17:59:20,850.850 INFO:teuthology.orchestra.console:Power off smithi062 2022-09-03T17:59:20.892 INFO:teuthology.kill:2022-09-03 17:59:20,891.891 INFO:teuthology.orchestra.console:Power off smithi087 2022-09-03T17:59:29.256 INFO:teuthology.kill:2022-09-03 17:59:29,256.256 INFO:teuthology.orchestra.console:Power off for smithi087 completed 2022-09-03T17:59:29.404 INFO:teuthology.kill:2022-09-03 17:59:29,404.404 INFO:teuthology.lock.ops:unlocked: smithi087.front.sepia.ceph.com 2022-09-03T17:59:33.364 INFO:teuthology.kill:2022-09-03 17:59:33,364.364 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2022-09-03T17:59:33.515 INFO:teuthology.kill:2022-09-03 17:59:33,515.515 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com 2022-09-03T18:01:33.859 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-03T18:01:33.965 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi087 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-03_14:45:48-rbd-wip-yuri-testing-2022-09-02-0945-quincy-distro-default-smithi/7009604