2021-12-30T03:06:17.969 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-30T03:06:18.085 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging 2021-12-30T03:06:18.086 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server 2021-12-30T03:06:18.086 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2021-12-30T03:06:18.105 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete 2021-12-30T03:06:18.105 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi168.front.sepia.ceph.com' reimaging 2021-12-30T03:06:18.105 INFO:teuthology.lock.ops:Updating [ubuntu@smithi168.front.sepia.ceph.com]: reset os type and version on server 2021-12-30T03:06:18.106 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server 2021-12-30T03:06:18.124 INFO:teuthology.lock.ops:Node 'ubuntu@smithi168.front.sepia.ceph.com' reimaging is complete 2021-12-30T03:06:18.317 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of centos 8.stream 2021-12-30T03:06:18.327 INFO:teuthology.provision.fog.smithi168:Scheduling deploy of centos 8.stream 2021-12-30T03:06:18.647 INFO:teuthology.orchestra.console:Power off smithi116 2021-12-30T03:06:18.662 INFO:teuthology.orchestra.console:Power off smithi168 2021-12-30T03:06:31.136 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2021-12-30T03:06:31.141 INFO:teuthology.orchestra.console:Power off for smithi168 completed 2021-12-30T03:06:31.237 INFO:teuthology.orchestra.console:Power on smithi116 2021-12-30T03:06:31.245 INFO:teuthology.orchestra.console:Power on smithi168 2021-12-30T03:06:35.494 INFO:teuthology.orchestra.console:Power on for smithi116 completed 2021-12-30T03:06:35.596 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2021-12-30T03:06:39.611 INFO:teuthology.orchestra.console:Power on for smithi168 completed 2021-12-30T03:06:39.713 INFO:teuthology.provision.fog.smithi168:Waiting for deploy to finish 2021-12-30T03:11:41.855 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-30T03:11:42.129 INFO:teuthology.provision.fog.smithi116:Node is ready 2021-12-30T03:11:42.146 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2021-12-30T03:11:42.203 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2021-12-30T03:11:42.533 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2021-12-30T03:12:08.676 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-30T03:12:33.814 INFO:teuthology.provision.fog.smithi168:Node is ready 2021-12-30T03:12:33.835 INFO:teuthology.orchestra.run.smithi168.stdout:smithi168.front.sepia.ceph.com 2021-12-30T03:12:33.894 INFO:teuthology.orchestra.run.smithi168.stdout:172.21.15.168 smithi168.front.sepia.ceph.com smithi168 2021-12-30T03:12:34.237 INFO:teuthology.provision.fog.smithi168:Deploy complete! 2021-12-30T03:12:34.388 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2021-12-30T03:12:34.389 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-30T03:12:34.407 INFO:teuthology.lock.ops:Checking smithi168.front.sepia.ceph.com 2021-12-30T03:12:34.408 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-30T03:12:34.425 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2021-12-30T03:12:34.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2021-12-30T03:12:34.617 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2021-12-30T03:12:34.674 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2021-12-30T03:12:34.675 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8" 2021-12-30T03:12:34.675 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2021-12-30T03:12:34.675 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2021-12-30T03:12:34.675 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8" 2021-12-30T03:12:34.676 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8" 2021-12-30T03:12:34.676 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-30T03:12:34.676 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2021-12-30T03:12:34.676 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-30T03:12:34.677 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2021-12-30T03:12:34.677 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-30T03:12:34.677 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-30T03:12:34.677 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-30T03:12:34.678 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2021-12-30T03:12:34.696 INFO:teuthology.lock.ops:Updating [smithi168.front.sepia.ceph.com]: set os type and version on server 2021-12-30T03:12:34.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi168.front.sepia.ceph.com' 2021-12-30T03:12:34.887 INFO:teuthology.orchestra.run.smithi168.stdout:x86_64 2021-12-30T03:12:34.942 INFO:teuthology.orchestra.run.smithi168.stdout:NAME="CentOS Stream" 2021-12-30T03:12:34.942 INFO:teuthology.orchestra.run.smithi168.stdout:VERSION="8" 2021-12-30T03:12:34.943 INFO:teuthology.orchestra.run.smithi168.stdout:ID="centos" 2021-12-30T03:12:34.943 INFO:teuthology.orchestra.run.smithi168.stdout:ID_LIKE="rhel fedora" 2021-12-30T03:12:34.943 INFO:teuthology.orchestra.run.smithi168.stdout:VERSION_ID="8" 2021-12-30T03:12:34.943 INFO:teuthology.orchestra.run.smithi168.stdout:PLATFORM_ID="platform:el8" 2021-12-30T03:12:34.944 INFO:teuthology.orchestra.run.smithi168.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-30T03:12:34.944 INFO:teuthology.orchestra.run.smithi168.stdout:ANSI_COLOR="0;31" 2021-12-30T03:12:34.944 INFO:teuthology.orchestra.run.smithi168.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-30T03:12:34.944 INFO:teuthology.orchestra.run.smithi168.stdout:HOME_URL="https://centos.org/" 2021-12-30T03:12:34.945 INFO:teuthology.orchestra.run.smithi168.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-30T03:12:34.945 INFO:teuthology.orchestra.run.smithi168.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-30T03:12:34.945 INFO:teuthology.orchestra.run.smithi168.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-30T03:12:34.946 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server 2021-12-30T03:12:35.033 INFO:teuthology.dispatcher.supervisor:Running job 6588384 2021-12-30T03:12:35.033 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-12-30_03:05:02-rgw-master-distro-default-smithi/6588384 --name teuthology-2021-12-30_03:05:02-rgw-master-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8.stream} tasks/test_multi} -- /home/teuthworker/archive/teuthology-2021-12-30_03:05:02-rgw-master-distro-default-smithi/6588384/orig.config.yaml 2021-12-30T03:12:35.036 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-12-30_03:05:02-rgw-master-distro-default-smithi/6588384 2021-12-30T03:12:35.037 INFO:teuthology.dispatcher.supervisor:Job PID: 1331 2021-12-30T03:12:35.037 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-30T15:12:54.961 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-30T15:12:55.191 INFO:teuthology.kill:Killing Pids: {1331} 2021-12-30T15:12:56.265 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2021-12-30T15:12:56.266 INFO:teuthology.task.internal:roles: ubuntu@smithi168.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2021-12-30T15:12:56.267 INFO:teuthology.misc:Compressing logs... 2021-12-30T15:12:56.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2021-12-30T15:12:56.453 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi168.front.sepia.ceph.com' 2021-12-30T15:12:56.825 INFO:teuthology.misc:Compressing logs... 2021-12-30T15:12:56.976 INFO:teuthology.misc:Compressing logs... 2021-12-30T15:19:54.545 INFO:teuthology.kill:No teuthology processes running 2021-12-30T15:19:55.616 INFO:teuthology.kill:Nuking machines: ['smithi116', 'smithi168'] 2021-12-30T15:19:56.542 INFO:teuthology.kill:2021-12-30 15:19:56,541.541 INFO:teuthology.nuke:targets: 2021-12-30T15:19:56.543 INFO:teuthology.kill: smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDrADded2hEAvdJuZvzp3RRor3vTcTrh8+fKID99Qw9bCLuV6R4y4lew6q9/20DDXg8e9dqELBl6Yj599N14qnNYg2Ur8rLmvfDB1Pboc2J3jrc1aVYW8+CjjuHdpVw3pbDQ1I0AviV1tM/hZjfJgkLEiFh0dYDFcTHwXJrnNRDwznlULWVuGJacjBUYgephVwb0zikUpCOTpF1Eb4TC4Y36afL75vXHstOuOpYHc42vpQIZNqbLeBrqhystrH9x+PTPoWXuQ6fkjJ44I3etf9r5583aMEfcTJXFwWalNmkdwwBtcgHwcZvstSpSXCxtmfdJ0rO66/3xhY95H04vT/YamYIKky2je959YfwVa7dTcOoZylQ7ZwVOPurkdGvIlImc/2XKLtLH3m3T45+5/x4OxXXCkivuNGqxOueSltvTXObEAQfytv7OvscocVIAiiReeNH+lXnMc/ZjPvu0VwpTyCqKRm0ZAsRRjQWNZjLF/1DwmGE7PDjJczcfunNGGk= 2021-12-30T15:19:56.543 INFO:teuthology.kill: smithi168.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWi6tFniEsL0UR5WTs38F8IQd/UvKyYpdNWc4JYEa6JeDdDlVYNNwB2lrFpN/0ZF1dlaHGEVnSmhGR3h4TeN74pXv9l/KXn/pLOV1a3lIoggQbxD4hT6O59hoL+odUcwKZ9xXwKlFnszhATN01SPhAAGRpGc0Va+75iUhbdlgtay6BKPIBOEmKT2ObBTBjQnPfUefIh9RxWfyy5Kfr1yK2GH5xxNPakMrHSYNuwt7QQiTUaBNMatoyYJmptyH211NbE0bQJbZ6v6jMpUYF8jDYwIV3Vk+I4B9kGdAxOKY9CTVX/6fsiK0ke5iFP1RwujvQxReVWhchDF/tz2cTv5hMpayyAjPPxWNvvSwl39oUZf6/e/hW+VhtS5Wq6kn5JGibwHa7hikHI2w13/i7FG+Ito9GGRdhtgJeqYTLuXjQykbq3rKE0zpRdxFMFpPpg1FkzplHdhpIJA2qE1Jlh8D/tK6ONpJxqBFJTidzXgvPwF/cjHZuVyRR1z8K3Vi3lCc= 2021-12-30T15:19:56.568 INFO:teuthology.kill:2021-12-30 15:19:56,567.567 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-30T15:19:56.571 INFO:teuthology.kill:2021-12-30 15:19:56,570.570 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-30T15:19:56.643 INFO:teuthology.kill:2021-12-30 15:19:56,643.643 INFO:teuthology.orchestra.console:Power off smithi116 2021-12-30T15:19:56.655 INFO:teuthology.kill:2021-12-30 15:19:56,655.655 INFO:teuthology.orchestra.console:Power off smithi168 2021-12-30T15:20:05.018 INFO:teuthology.kill:2021-12-30 15:20:05,017.017 INFO:teuthology.orchestra.console:Power off for smithi168 completed 2021-12-30T15:20:05.177 INFO:teuthology.kill:2021-12-30 15:20:05,175.175 INFO:teuthology.lock.ops:unlocked: smithi168.front.sepia.ceph.com 2021-12-30T15:20:09.123 INFO:teuthology.kill:2021-12-30 15:20:09,123.123 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2021-12-30T15:20:09.270 INFO:teuthology.kill:2021-12-30 15:20:09,269.269 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com 2021-12-30T15:22:09.557 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15