2022-04-10T01:03:57.206 INFO:root:teuthology version: 1.1.0-a347b2ac
2022-04-10T01:03:57.348 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi005.front.sepia.ceph.com' reimaging
2022-04-10T01:03:57.349 INFO:teuthology.lock.ops:Updating [ubuntu@smithi005.front.sepia.ceph.com]: reset os type and version on server
2022-04-10T01:03:57.349 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2022-04-10T01:03:57.370 INFO:teuthology.lock.ops:Node 'ubuntu@smithi005.front.sepia.ceph.com' reimaging is complete
2022-04-10T01:03:57.370 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi043.front.sepia.ceph.com' reimaging
2022-04-10T01:03:57.371 INFO:teuthology.lock.ops:Updating [ubuntu@smithi043.front.sepia.ceph.com]: reset os type and version on server
2022-04-10T01:03:57.371 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server
2022-04-10T01:03:57.390 INFO:teuthology.lock.ops:Node 'ubuntu@smithi043.front.sepia.ceph.com' reimaging is complete
2022-04-10T01:03:57.588 INFO:teuthology.provision.fog.smithi005:Scheduling deploy of centos 8.stream
2022-04-10T01:03:57.626 INFO:teuthology.provision.fog.smithi043:Scheduling deploy of centos 8.stream
2022-04-10T01:03:57.946 INFO:teuthology.orchestra.console:Power off smithi005
2022-04-10T01:03:57.985 INFO:teuthology.orchestra.console:Power off smithi043
2022-04-10T01:03:58.095 INFO:teuthology.orchestra.console:Power off for smithi005 completed
2022-04-10T01:03:58.129 INFO:teuthology.orchestra.console:Power off for smithi043 completed
2022-04-10T01:03:58.197 INFO:teuthology.orchestra.console:Power on smithi005
2022-04-10T01:03:58.231 INFO:teuthology.orchestra.console:Power on smithi043
2022-04-10T01:04:02.482 INFO:teuthology.orchestra.console:Power on for smithi043 completed
2022-04-10T01:04:02.583 INFO:teuthology.provision.fog.smithi043:Waiting for deploy to finish
2022-04-10T01:04:06.564 INFO:teuthology.orchestra.console:Power on for smithi005 completed
2022-04-10T01:04:06.666 INFO:teuthology.provision.fog.smithi005:Waiting for deploy to finish
2022-04-10T01:06:22.777 ERROR:teuthology.orchestra.connection:Error authenticating with smithi005.front.sepia.ceph.com: Authentication failed.
2022-04-10T01:08:16.647 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-10T01:08:16.956 INFO:teuthology.provision.fog.smithi043:Node is ready
2022-04-10T01:08:16.976 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043.front.sepia.ceph.com
2022-04-10T01:08:17.034 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43 smithi043.front.sepia.ceph.com smithi043
2022-04-10T01:08:17.405 INFO:teuthology.provision.fog.smithi043:Deploy complete!
2022-04-10T01:08:21.727 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-10T01:08:22.060 INFO:teuthology.provision.fog.smithi005:Node is ready
2022-04-10T01:08:22.079 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005.front.sepia.ceph.com
2022-04-10T01:08:22.138 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5 smithi005.front.sepia.ceph.com smithi005
2022-04-10T01:08:22.516 INFO:teuthology.provision.fog.smithi005:Deploy complete!
2022-04-10T01:08:22.706 INFO:teuthology.lock.ops:Checking smithi005.front.sepia.ceph.com
2022-04-10T01:08:22.707 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-10T01:08:22.727 INFO:teuthology.lock.ops:Checking smithi043.front.sepia.ceph.com
2022-04-10T01:08:22.728 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-10T01:08:22.748 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: set os type and version on server
2022-04-10T01:08:22.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com'
2022-04-10T01:08:22.937 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64
2022-04-10T01:08:22.995 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream"
2022-04-10T01:08:22.996 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8"
2022-04-10T01:08:22.996 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos"
2022-04-10T01:08:22.996 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora"
2022-04-10T01:08:22.996 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8"
2022-04-10T01:08:22.997 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8"
2022-04-10T01:08:22.997 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-10T01:08:22.997 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31"
2022-04-10T01:08:22.997 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-10T01:08:22.997 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/"
2022-04-10T01:08:22.998 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-10T01:08:22.998 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-10T01:08:22.998 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-10T01:08:22.999 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2022-04-10T01:08:23.018 INFO:teuthology.lock.ops:Updating [smithi043.front.sepia.ceph.com]: set os type and version on server
2022-04-10T01:08:23.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com'
2022-04-10T01:08:23.214 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64
2022-04-10T01:08:23.273 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream"
2022-04-10T01:08:23.274 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="8"
2022-04-10T01:08:23.274 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos"
2022-04-10T01:08:23.274 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora"
2022-04-10T01:08:23.275 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="8"
2022-04-10T01:08:23.275 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el8"
2022-04-10T01:08:23.275 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-10T01:08:23.275 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31"
2022-04-10T01:08:23.275 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-10T01:08:23.276 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/"
2022-04-10T01:08:23.276 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-10T01:08:23.276 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-10T01:08:23.276 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-10T01:08:23.277 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server
2022-04-10T01:08:23.385 INFO:teuthology.dispatcher.supervisor:Running job 6784449
2022-04-10T01:08:23.386 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a347b2ac5400a43ba76a678399b1deda7dae1a6c/virtualenv/bin/teuthology -v --owner scheduled_mrfreezeex@teuthology --archive /home/teuthworker/archive/mrfreezeex-2022-04-09_15:53:21-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6784449 --name mrfreezeex-2022-04-09_15:53:21-rbd-wip-mrfreezeex-multipeer-distro-default-smithi --description rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-hybrid policy/simple supported-random-distro$/{centos_8}} -- /home/teuthworker/archive/mrfreezeex-2022-04-09_15:53:21-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6784449/orig.config.yaml
2022-04-10T01:08:23.389 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mrfreezeex-2022-04-09_15:53:21-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6784449
2022-04-10T01:08:23.390 INFO:teuthology.dispatcher.supervisor:Job PID: 3112
2022-04-10T01:08:23.390 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-10T07:38:34.123 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-04-10T07:38:34.394 INFO:teuthology.kill:Killing Pids: {3112}
2022-04-10T07:38:35.657 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0']
2022-04-10T07:38:35.658 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']
2022-04-10T07:38:35.658 INFO:teuthology.misc:Compressing logs...
2022-04-10T07:38:35.659 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com'
2022-04-10T07:38:35.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com'
2022-04-10T07:38:36.304 INFO:teuthology.misc:Compressing logs...
2022-04-10T07:39:13.114 INFO:teuthology.misc:Compressing logs...
2022-04-10T07:43:29.924 INFO:teuthology.kill:No teuthology processes running
2022-04-10T07:43:31.121 INFO:teuthology.kill:Nuking machines: ['smithi005', 'smithi043']
2022-04-10T07:43:32.077 INFO:teuthology.kill:2022-04-10 07:43:32,077.077 INFO:teuthology.nuke:targets:
2022-04-10T07:43:32.078 INFO:teuthology.kill:  smithi005.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6a7IpOZPtZwIOOYNtsNKq1njcAjwzrK01ZhS9QRS+ifxFUZFe8h8edywJ1zqQAOZRNH+E+aKD6C1F+nyv2HdWrjIXqbyTlgyYxU/VS9RHc27G2rFDS9p7aogFlkCDXHKfx+BiYqnbGBmqtGKL0Os0Ur7m1uZ0hn34qmSiZS0XD4/wSTSwW6XwhkJCMxuMaoWzthqmojoClGBbutqSDhmOsLBOf3g+7PchS3HOPGYIUJ+rd9p2lkM+3YV76Lo/9Sj5TrYxONbGx8tZEnw+xkYwrGqTbxTAax8DX7Bws8jRXQeKekmgA1m+JzxmWB7vTIx2Nm9wo0iMLWQT53nbhx2Mcjg224BSYyYvQiJnPX46yv25nTVAzLnVXQ8lxTUXmacy2qwHRyOfe8BNYCisGmU5Z9suX/pFgnFEBbzf5wcFmW/KwsuTDa/A2nzRNRpsaWBv53ZA7jNXBXIYOYoaV3GFYLA2AdhvsGTMzpXYxa9f+QUxhJWFvLiDmBJk0kcwhCk=
2022-04-10T07:43:32.078 INFO:teuthology.kill:  smithi043.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxbBDJqyiaOu9Bbtq4sC6enMBXqfpHtMB96N/mf2lA/aaKGlTMNpp2Zye0gQ9z4HygetquzjtuApbZg+ldNp1ZyvL7lcYvBos5rNC2+zTYmhk6+9bvHmSv/51wk67aheDavAy1QvcABfu4F/qTSGfqoWAXv+BZXYH2KjS8M7Rrp9YbBjcoGRsGwwvAzHLqcK806WaVGWOJCdfcpsKWVrL3R/MbUZRVhvsi8PvAeGzMAnpsYO5nyNH2Syla720zcmld++unoK06P4fWkF8hFOq/ooREjY6oH+R5zVohi1pDrEAviod6TLCvAhauFyhBgT7bcCvLk2P3/VvC7CkiX17A4fLZYnMZpQHfgsSLvlbrLFgF767L2r+/ekL114Zhxc4EolKhuZlEodiLAvt3LVgb3FcGq+J68W/ejuEY9nmZEl4pfa/jsq5LVTFoBwjcpJd40J8QOcPMteeniG7rH77A+ScsxTVCvAhiSiLpwxNfZjU6FkwakxUJq+3gEjzy21s=
2022-04-10T07:43:32.104 INFO:teuthology.kill:2022-04-10 07:43:32,103.103 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-10T07:43:32.106 INFO:teuthology.kill:2022-04-10 07:43:32,106.106 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-10T07:43:32.184 INFO:teuthology.kill:2022-04-10 07:43:32,184.184 INFO:teuthology.orchestra.console:Power off smithi043
2022-04-10T07:43:32.198 INFO:teuthology.kill:2022-04-10 07:43:32,198.198 INFO:teuthology.orchestra.console:Power off smithi005
2022-04-10T07:43:40.556 INFO:teuthology.kill:2022-04-10 07:43:40,555.555 INFO:teuthology.orchestra.console:Power off for smithi043 completed
2022-04-10T07:43:40.561 INFO:teuthology.kill:2022-04-10 07:43:40,561.561 INFO:teuthology.orchestra.console:Power off for smithi005 completed
2022-04-10T07:43:40.705 INFO:teuthology.kill:2022-04-10 07:43:40,704.704 INFO:teuthology.lock.ops:unlocked: smithi043.front.sepia.ceph.com
2022-04-10T07:43:40.711 INFO:teuthology.kill:2022-04-10 07:43:40,711.711 INFO:teuthology.lock.ops:unlocked: smithi005.front.sepia.ceph.com
2022-04-10T07:45:40.929 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-10T07:45:41.046 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi005 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-09_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6784252
2022-04-10T07:45:41.046 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi043 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-09_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6784252