2022-07-18T02:48:45.086 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5 2022-07-18T02:48:45.374 INFO:teuthology.lock.ops:Start node 'smithi133.front.sepia.ceph.com' reimaging 2022-07-18T02:48:45.376 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: reset os type and version on server 2022-07-18T02:48:45.376 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-07-18T02:48:45.434 INFO:teuthology.lock.ops:Node 'smithi133.front.sepia.ceph.com' reimaging is complete 2022-07-18T02:48:45.468 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging 2022-07-18T02:48:45.477 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server 2022-07-18T02:48:45.487 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-07-18T02:48:45.530 INFO:teuthology.lock.ops:Node 'smithi159.front.sepia.ceph.com' reimaging is complete 2022-07-18T02:48:45.789 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of centos 8.stream 2022-07-18T02:48:45.804 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 8.stream 2022-07-18T02:48:46.221 INFO:teuthology.orchestra.console:Power off smithi133 2022-07-18T02:48:46.372 INFO:teuthology.orchestra.console:Power off smithi159 2022-07-18T02:48:54.827 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-07-18T02:48:54.929 INFO:teuthology.orchestra.console:Power on smithi159 2022-07-18T02:48:58.878 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2022-07-18T02:48:58.982 INFO:teuthology.orchestra.console:Power on smithi133 2022-07-18T02:49:03.370 INFO:teuthology.orchestra.console:Power on for smithi159 completed 2022-07-18T02:49:03.480 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2022-07-18T02:49:07.386 INFO:teuthology.orchestra.console:Power on for smithi133 completed 2022-07-18T02:49:07.488 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish 2022-07-18T02:53:45.942 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-18T02:53:46.228 INFO:teuthology.provision.fog.smithi133:Node is ready 2022-07-18T02:53:46.249 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com 2022-07-18T02:53:46.309 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133 2022-07-18T02:53:46.671 INFO:teuthology.provision.fog.smithi133:Deploy complete! 2022-07-18T02:54:00.356 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-18T02:54:00.617 INFO:teuthology.provision.fog.smithi159:Node is ready 2022-07-18T02:54:00.637 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2022-07-18T02:54:00.697 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2022-07-18T02:54:01.053 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2022-07-18T02:54:01.232 INFO:teuthology.lock.ops:Checking smithi133.front.sepia.ceph.com 2022-07-18T02:54:01.233 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-18T02:54:01.255 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2022-07-18T02:54:01.256 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-18T02:54:01.275 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: set os type and version on server 2022-07-18T02:54:01.276 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-07-18T02:54:01.544 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2022-07-18T02:54:01.603 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2022-07-18T02:54:01.603 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8" 2022-07-18T02:54:01.604 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2022-07-18T02:54:01.604 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2022-07-18T02:54:01.604 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8" 2022-07-18T02:54:01.604 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2022-07-18T02:54:01.605 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-18T02:54:01.605 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2022-07-18T02:54:01.605 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-18T02:54:01.605 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2022-07-18T02:54:01.606 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-18T02:54:01.606 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-18T02:54:01.606 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-18T02:54:01.607 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-07-18T02:54:01.633 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2022-07-18T02:54:01.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-07-18T02:54:01.844 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2022-07-18T02:54:01.903 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2022-07-18T02:54:01.903 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8" 2022-07-18T02:54:01.904 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2022-07-18T02:54:01.904 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2022-07-18T02:54:01.905 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8" 2022-07-18T02:54:01.905 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2022-07-18T02:54:01.906 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-18T02:54:01.906 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2022-07-18T02:54:01.908 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-18T02:54:01.908 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2022-07-18T02:54:01.909 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-18T02:54:01.909 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-18T02:54:01.910 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-18T02:54:01.911 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-07-18T02:54:02.024 INFO:teuthology.dispatcher.supervisor:Running job 6935300 2022-07-18T02:54:02.024 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/bin/teuthology -v --owner scheduled_mbenjamin@shatner --archive /home/teuthworker/archive/mbenjamin-2022-07-17_22:24:15-rgw-wip-rgwlc-azone-distro-default-smithi/6935300 --name mbenjamin-2022-07-17_22:24:15-rgw-wip-rgwlc-azone-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi} -- /home/teuthworker/archive/mbenjamin-2022-07-17_22:24:15-rgw-wip-rgwlc-azone-distro-default-smithi/6935300/orig.config.yaml 2022-07-18T02:54:02.034 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mbenjamin-2022-07-17_22:24:15-rgw-wip-rgwlc-azone-distro-default-smithi/6935300 2022-07-18T02:54:02.035 INFO:teuthology.dispatcher.supervisor:Job PID: 23179 2022-07-18T02:54:02.035 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-18T14:54:58.627 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-18T14:54:58.826 INFO:teuthology.kill:Killing Pids: {23179} 2022-07-18T14:55:00.140 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-07-18T14:55:00.141 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-07-18T14:55:00.141 INFO:teuthology.misc:Compressing logs... 2022-07-18T14:55:00.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-07-18T14:55:00.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-07-18T14:55:00.949 INFO:teuthology.misc:Compressing logs... 2022-07-18T14:55:01.096 INFO:teuthology.misc:Compressing logs... 2022-07-18T15:07:20.062 INFO:teuthology.kill:No teuthology processes running 2022-07-18T15:07:21.416 INFO:teuthology.kill:Nuking machines: ['smithi133', 'smithi159'] 2022-07-18T15:07:22.468 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-18T15:07:22.469 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-18T15:07:22.470 INFO:teuthology.kill:2022-07-18 15:07:22,468.468 INFO:teuthology.nuke:targets: 2022-07-18T15:07:22.470 INFO:teuthology.kill: smithi133.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbz643aJzTA1MGouCBCaX2wAlr7gxhsuzbiWpPfB1J+XwHbAt+kDOGp2Ft/jOzaXFKrq9e20ENTcd5lzC730MsuhxkXAFpXApXKyIfdGdowsKOGictcs22h7dXfvGGl7lytZg65AHT5kAGQZkADb24rTTvlM9ZtQrM2wLMo5pHBM4p+2nlHFEMH4NvWM0W4qQ+MUTv+V9EbLfB3Kf4OJcfMuw4S9+fSDf3bqVb8I7WIx/wtYFxEEJT4PolE/dvR7kCe/C6rqqqDMP154xs5/17c652VftNZq41Vv8IueKmd3wq2ZOM2zqrNLCGr1FkE7T4jBC89zWOCn9PRu+RF9Vl0lPLKOrkVOVOoQdEZSZixPiOEkRyKEkWNxkyIPtS/EDTjjDKIZSrCeRLm3EOXMP0eFTEa98qWrsoaxtLOeDIPM3HwufNX5YU5DLyvmx+x9ZQilp3QwJD89g+dzQqSSdDgxjTvJ2ZDvid4MkWd8G5Faw/UCUtHNq8FLc9BCGZ+pM= 2022-07-18T15:07:22.470 INFO:teuthology.kill: smithi159.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD0XX/4wSzWw+RBIBAqfnofJBXzIjtevgLHZtgd1wszCkZ2NrfTAt7OZWuj/Imft/uz6GkoqE0v5k/VRYEThvU4Fg0FCIKaBPOc7rZOR3Ego4Qk+Xun+Fm0AfDMCVtZTsCY/dImi9Kh2bOruLyGZnRPD4hnLoYNgsHioSMJjGOyWj5cIzGU7u/UNmmfSDvQqTnBUbvXYWLuRkwYDnyUGt36BVOM+OHhS3ODwnKXtNycReRYuxdoh8T4Q193sdYXaaISZxtQnPiIG/tRMy7vDanx5m+m9I0i65kh/J4umQQuqQ/DutpXO8TrPXAxqYtkyloh9IrBRV1bwN55ziIA9cPLHl9LBFA7cGKjk/2Bm5tEz9DxeMWUPygu6jFc+UUh3zNTaaJp9nSPMYmwYP8Ub7TmEluVDqPJh6jctHSYllsA/scLg19rIGTi7E1s7KxLOu8bBT++U9T6HPeVXqwZqZlLmEoIBKTE2wnIS5NHj/M/pdzfAvd/OqdUsOJGxMTJHDc= 2022-07-18T15:07:22.471 INFO:teuthology.kill:2022-07-18 15:07:22,468.468 INFO:teuthology.nuke:Checking targets against current locks 2022-07-18T15:07:22.519 INFO:teuthology.kill:2022-07-18 15:07:22,519.519 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-18T15:07:22.549 INFO:teuthology.kill:2022-07-18 15:07:22,549.549 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-18T15:07:22.607 INFO:teuthology.kill:2022-07-18 15:07:22,606.606 INFO:teuthology.orchestra.console:Power off smithi133 2022-07-18T15:07:22.626 INFO:teuthology.kill:2022-07-18 15:07:22,626.626 INFO:teuthology.orchestra.console:Power off smithi159 2022-07-18T15:07:30.978 INFO:teuthology.kill:2022-07-18 15:07:30,978.978 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2022-07-18T15:07:30.983 INFO:teuthology.kill:2022-07-18 15:07:30,983.983 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-07-18T15:07:31.131 INFO:teuthology.kill:2022-07-18 15:07:31,130.130 INFO:teuthology.lock.ops:unlocked: smithi133.front.sepia.ceph.com 2022-07-18T15:07:31.135 INFO:teuthology.kill:2022-07-18 15:07:31,135.135 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com 2022-07-18T15:09:31.353 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15