2021-10-02T07:59:18.279 INFO:root:teuthology version: 1.1.0-f08b273e 2021-10-02T07:59:18.385 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba018.front.sepia.ceph.com' reimaging 2021-10-02T07:59:18.386 INFO:teuthology.lock.ops:Updating [ubuntu@gibba018.front.sepia.ceph.com]: reset os type and version on server 2021-10-02T07:59:18.387 INFO:teuthology.lock.ops:Updating gibba018.front.sepia.ceph.com on lock server 2021-10-02T07:59:18.404 INFO:teuthology.lock.ops:Node 'ubuntu@gibba018.front.sepia.ceph.com' reimaging is complete 2021-10-02T07:59:18.404 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba022.front.sepia.ceph.com' reimaging 2021-10-02T07:59:18.405 INFO:teuthology.lock.ops:Updating [ubuntu@gibba022.front.sepia.ceph.com]: reset os type and version on server 2021-10-02T07:59:18.405 INFO:teuthology.lock.ops:Updating gibba022.front.sepia.ceph.com on lock server 2021-10-02T07:59:18.423 INFO:teuthology.lock.ops:Node 'ubuntu@gibba022.front.sepia.ceph.com' reimaging is complete 2021-10-02T07:59:18.629 INFO:teuthology.provision.fog.gibba022:Scheduling deploy of rhel 8.4 2021-10-02T07:59:18.632 INFO:teuthology.provision.fog.gibba018:Scheduling deploy of rhel 8.4 2021-10-02T07:59:18.952 INFO:teuthology.orchestra.console:Power off gibba018 2021-10-02T07:59:18.966 INFO:teuthology.orchestra.console:Power off gibba022 2021-10-02T07:59:19.101 INFO:teuthology.orchestra.console:Power off for gibba018 completed 2021-10-02T07:59:19.202 INFO:teuthology.orchestra.console:Power on gibba018 2021-10-02T07:59:27.342 INFO:teuthology.orchestra.console:Power off for gibba022 completed 2021-10-02T07:59:27.444 INFO:teuthology.orchestra.console:Power on gibba022 2021-10-02T07:59:27.577 INFO:teuthology.orchestra.console:Power on for gibba018 completed 2021-10-02T07:59:27.678 INFO:teuthology.provision.fog.gibba018:Waiting for deploy to finish 2021-10-02T07:59:35.818 INFO:teuthology.orchestra.console:Power on for gibba022 completed 2021-10-02T07:59:35.919 INFO:teuthology.provision.fog.gibba022:Waiting for deploy to finish 2021-10-02T08:03:50.623 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-10-02T08:03:50.848 INFO:teuthology.provision.fog.gibba018:Node is ready 2021-10-02T08:03:50.859 INFO:teuthology.orchestra.run.gibba018.stdout:gibba018.front.sepia.ceph.com 2021-10-02T08:03:50.912 INFO:teuthology.orchestra.run.gibba018.stdout:172.21.2.118 gibba018.front.sepia.ceph.com gibba018 2021-10-02T08:03:51.240 INFO:teuthology.provision.fog.gibba018:Deploy complete! 2021-10-02T08:04:07.807 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-10-02T08:04:08.008 INFO:teuthology.provision.fog.gibba022:Node is ready 2021-10-02T08:04:08.019 INFO:teuthology.orchestra.run.gibba022.stdout:gibba022.front.sepia.ceph.com 2021-10-02T08:04:08.071 INFO:teuthology.orchestra.run.gibba022.stdout:172.21.2.122 gibba022.front.sepia.ceph.com gibba022 2021-10-02T08:04:08.365 INFO:teuthology.provision.fog.gibba022:Deploy complete! 2021-10-02T08:04:08.501 INFO:teuthology.lock.ops:Checking gibba018.front.sepia.ceph.com 2021-10-02T08:04:08.502 INFO:teuthology.lock.ops:New key found. Updating... 2021-10-02T08:04:08.519 INFO:teuthology.lock.ops:Checking gibba022.front.sepia.ceph.com 2021-10-02T08:04:08.519 INFO:teuthology.lock.ops:New key found. Updating... 2021-10-02T08:04:08.535 INFO:teuthology.lock.ops:Updating [gibba018.front.sepia.ceph.com]: set os type and version on server 2021-10-02T08:04:08.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba018.front.sepia.ceph.com' 2021-10-02T08:04:08.741 INFO:teuthology.orchestra.run.gibba018.stdout:x86_64 2021-10-02T08:04:08.793 INFO:teuthology.orchestra.run.gibba018.stdout:NAME="Red Hat Enterprise Linux" 2021-10-02T08:04:08.793 INFO:teuthology.orchestra.run.gibba018.stdout:VERSION="8.4 (Ootpa)" 2021-10-02T08:04:08.794 INFO:teuthology.orchestra.run.gibba018.stdout:ID="rhel" 2021-10-02T08:04:08.794 INFO:teuthology.orchestra.run.gibba018.stdout:ID_LIKE="fedora" 2021-10-02T08:04:08.794 INFO:teuthology.orchestra.run.gibba018.stdout:VERSION_ID="8.4" 2021-10-02T08:04:08.794 INFO:teuthology.orchestra.run.gibba018.stdout:PLATFORM_ID="platform:el8" 2021-10-02T08:04:08.794 INFO:teuthology.orchestra.run.gibba018.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-10-02T08:04:08.795 INFO:teuthology.orchestra.run.gibba018.stdout:ANSI_COLOR="0;31" 2021-10-02T08:04:08.795 INFO:teuthology.orchestra.run.gibba018.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-10-02T08:04:08.795 INFO:teuthology.orchestra.run.gibba018.stdout:HOME_URL="https://www.redhat.com/" 2021-10-02T08:04:08.795 INFO:teuthology.orchestra.run.gibba018.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-10-02T08:04:08.796 INFO:teuthology.orchestra.run.gibba018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-10-02T08:04:08.796 INFO:teuthology.orchestra.run.gibba018.stdout: 2021-10-02T08:04:08.796 INFO:teuthology.orchestra.run.gibba018.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-10-02T08:04:08.796 INFO:teuthology.orchestra.run.gibba018.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-10-02T08:04:08.796 INFO:teuthology.orchestra.run.gibba018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-10-02T08:04:08.797 INFO:teuthology.orchestra.run.gibba018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-10-02T08:04:08.798 INFO:teuthology.lock.ops:Updating gibba018.front.sepia.ceph.com on lock server 2021-10-02T08:04:08.815 INFO:teuthology.lock.ops:Updating [gibba022.front.sepia.ceph.com]: set os type and version on server 2021-10-02T08:04:08.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba022.front.sepia.ceph.com' 2021-10-02T08:04:09.050 INFO:teuthology.orchestra.run.gibba022.stdout:x86_64 2021-10-02T08:04:09.102 INFO:teuthology.orchestra.run.gibba022.stdout:NAME="Red Hat Enterprise Linux" 2021-10-02T08:04:09.102 INFO:teuthology.orchestra.run.gibba022.stdout:VERSION="8.4 (Ootpa)" 2021-10-02T08:04:09.102 INFO:teuthology.orchestra.run.gibba022.stdout:ID="rhel" 2021-10-02T08:04:09.102 INFO:teuthology.orchestra.run.gibba022.stdout:ID_LIKE="fedora" 2021-10-02T08:04:09.103 INFO:teuthology.orchestra.run.gibba022.stdout:VERSION_ID="8.4" 2021-10-02T08:04:09.103 INFO:teuthology.orchestra.run.gibba022.stdout:PLATFORM_ID="platform:el8" 2021-10-02T08:04:09.103 INFO:teuthology.orchestra.run.gibba022.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-10-02T08:04:09.103 INFO:teuthology.orchestra.run.gibba022.stdout:ANSI_COLOR="0;31" 2021-10-02T08:04:09.103 INFO:teuthology.orchestra.run.gibba022.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-10-02T08:04:09.104 INFO:teuthology.orchestra.run.gibba022.stdout:HOME_URL="https://www.redhat.com/" 2021-10-02T08:04:09.104 INFO:teuthology.orchestra.run.gibba022.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-10-02T08:04:09.104 INFO:teuthology.orchestra.run.gibba022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-10-02T08:04:09.104 INFO:teuthology.orchestra.run.gibba022.stdout: 2021-10-02T08:04:09.104 INFO:teuthology.orchestra.run.gibba022.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-10-02T08:04:09.105 INFO:teuthology.orchestra.run.gibba022.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-10-02T08:04:09.105 INFO:teuthology.orchestra.run.gibba022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-10-02T08:04:09.105 INFO:teuthology.orchestra.run.gibba022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-10-02T08:04:09.106 INFO:teuthology.lock.ops:Updating gibba022.front.sepia.ceph.com on lock server 2021-10-02T08:04:09.188 INFO:teuthology.dispatcher.supervisor:Running job 6408629 2021-10-02T08:04:09.188 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-09-27_02:01:03-rbd-master-distro-basic-gibba/6408629 --name teuthology-2021-09-27_02:01:03-rbd-master-distro-basic-gibba --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} -- /home/teuthworker/archive/teuthology-2021-09-27_02:01:03-rbd-master-distro-basic-gibba/6408629/orig.config.yaml 2021-10-02T08:04:09.192 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-09-27_02:01:03-rbd-master-distro-basic-gibba/6408629 2021-10-02T08:04:09.193 INFO:teuthology.dispatcher.supervisor:Job PID: 12807 2021-10-02T08:04:09.193 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-10-02T20:05:01.755 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-10-02T20:05:01.963 INFO:teuthology.kill:Killing Pids: {12807} 2021-10-02T20:05:03.224 INFO:teuthology.task.internal:roles: ubuntu@gibba018.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2021-10-02T20:05:03.225 INFO:teuthology.task.internal:roles: ubuntu@gibba022.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2021-10-02T20:05:03.225 INFO:teuthology.misc:Compressing logs... 2021-10-02T20:05:03.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba018.front.sepia.ceph.com' 2021-10-02T20:05:03.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@gibba022.front.sepia.ceph.com' 2021-10-02T20:05:03.811 INFO:teuthology.misc:Compressing logs... 2021-10-02T20:05:38.152 INFO:teuthology.misc:Compressing logs... 2021-10-02T20:06:48.618 INFO:teuthology.orchestra.run.gibba022.stderr:gzip: /var/log/ceph/cluster1-client.mirror.3.39174.log: file size changed while zipping 2021-10-02T20:07:40.973 INFO:teuthology.kill:No teuthology processes running 2021-10-02T20:07:42.173 INFO:teuthology.kill:Nuking machines: ['gibba018', 'gibba022'] 2021-10-02T20:07:43.144 INFO:teuthology.kill:2021-10-02 20:07:43,144.144 INFO:teuthology.nuke:targets: 2021-10-02T20:07:43.146 INFO:teuthology.kill: gibba018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvlmNevXJPPgdPw5tNSaV7Zy/hUrgr4I1l+oxYa73XPI1DCA7Mz8SlcYIvpwsHOy5iUKB1UBMFNBEnYALzG0BmTchN+fmDbKf2LlR82e/qef4TewnAcZ1k983HAiUnApGMX0pGeVi0YW+YFa9TJIofhtZJ3MXPK2ThnRUxTsR04AAqM+s/m1bY8dFI4NntPyaupT2e1h0jh5iCXDdIYMC7XVzCFsq0j0wx5ZABnNC3YQ/ClivwVDkSQ1YdgNHnbCg0v9jYVf9CTzqAI1EVXiJh24h0xyrNkmzaFf0FXnhtP6Gm3lx865sg1zF0m5GTXD0lP07Z0ZjjYXec0tSm8B/g0SqIyjxHWIbNTT1LQvNjn+hWBnZJHRtD5YmR1wY2T854V0YGSYsQcpahAvrHRv+R+jMz2QqDdErcbM7IxIwhzPP4PSUQwOeSKCBK1gLetPRwzWmUFWVnpHB7jxSUBOe1tqV/+gnDt/1VNbupGcb3jsareISogNK9wEZkekjfxak= 2021-10-02T20:07:43.146 INFO:teuthology.kill: gibba022.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwYk2WAnA+R6nOhhkRiXIrq2/ilzBwMMP1M9gdUpcuaH1AtnhqH6whohal6/wmYJc5juE/7BMVs//g/TxYeRsCVgv/kYipsqHCQhIntuLGaAdySo7ZY7B4GxgnkMI5C3UFB4PF2SweOE+IENys0/15GLCiCPVN4W/+Q2F9T/qHFiZ6K+cCqztytD1o1PA3H+PAu82RJMhF98p52iWCTGjWvzHJRVfUhNVrADfEDRrp3nnVD07y9s4VnqKzkuOZGb3h/yGujHGZb2JXFRAhE0qX2YM1z4VVwYxsbhli5GXxe3JRENItuptCEKvCfNt5mhLeHYSKLy5PEw9sC9pw8LDJq6N3kZWUihkil3uCPqaaC2Spv/272H8yrUfhEBMyTWhiUxGYSbjiCsgujIOen1C9lX+wYrrGH1YoWhfpFleukUFKNzU7kQ2KWxIr3QlzL4srP7sIUzIA6oLDNLyqPDtX4SNdvnzDUoYpdsdllLYyphNCnR0UO2rvU9ck471x/oE= 2021-10-02T20:07:43.170 INFO:teuthology.kill:2021-10-02 20:07:43,170.170 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-10-02T20:07:43.173 INFO:teuthology.kill:2021-10-02 20:07:43,173.173 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-10-02T20:07:43.242 INFO:teuthology.kill:2021-10-02 20:07:43,242.242 INFO:teuthology.orchestra.console:Power off gibba022 2021-10-02T20:07:43.257 INFO:teuthology.kill:2021-10-02 20:07:43,256.256 INFO:teuthology.orchestra.console:Power off gibba018 2021-10-02T20:07:51.621 INFO:teuthology.kill:2021-10-02 20:07:51,620.620 INFO:teuthology.orchestra.console:Power off for gibba022 completed 2021-10-02T20:07:51.626 INFO:teuthology.kill:2021-10-02 20:07:51,626.626 INFO:teuthology.orchestra.console:Power off for gibba018 completed 2021-10-02T20:07:51.793 INFO:teuthology.kill:2021-10-02 20:07:51,792.792 INFO:teuthology.lock.ops:unlocked: gibba022.front.sepia.ceph.com 2021-10-02T20:07:51.794 INFO:teuthology.kill:2021-10-02 20:07:51,794.794 INFO:teuthology.lock.ops:unlocked: gibba018.front.sepia.ceph.com 2021-10-02T20:09:52.140 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-10-02T20:09:52.246 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-10-02T20:09:52.247 INFO:teuthology.nuke:Checking targets against current locks 2021-10-02T20:09:52.348 INFO:teuthology.nuke:Not nuking gibba018.front.sepia.ceph.com because description doesn't match 2021-10-02T20:09:52.349 INFO:teuthology.nuke:Not nuking gibba022.front.sepia.ceph.com because description doesn't match