2022-02-17T14:09:49.037 INFO:root:teuthology version: 1.1.0-c6442aa9 2022-02-17T14:09:49.165 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging 2022-02-17T14:09:49.167 INFO:teuthology.lock.ops:Updating [ubuntu@smithi052.front.sepia.ceph.com]: reset os type and version on server 2022-02-17T14:09:49.167 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2022-02-17T14:09:49.187 INFO:teuthology.lock.ops:Node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging is complete 2022-02-17T14:09:49.187 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging 2022-02-17T14:09:49.188 INFO:teuthology.lock.ops:Updating [ubuntu@smithi138.front.sepia.ceph.com]: reset os type and version on server 2022-02-17T14:09:49.188 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-02-17T14:09:49.208 INFO:teuthology.lock.ops:Node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging is complete 2022-02-17T14:09:49.402 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of centos 8.stream 2022-02-17T14:09:49.427 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of centos 8.stream 2022-02-17T14:09:49.777 INFO:teuthology.orchestra.console:Power off smithi138 2022-02-17T14:09:49.793 INFO:teuthology.orchestra.console:Power off smithi052 2022-02-17T14:09:49.924 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-02-17T14:09:49.941 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2022-02-17T14:09:50.025 INFO:teuthology.orchestra.console:Power on smithi138 2022-02-17T14:09:50.042 INFO:teuthology.orchestra.console:Power on smithi052 2022-02-17T14:09:54.287 INFO:teuthology.orchestra.console:Power on for smithi138 completed 2022-02-17T14:09:54.298 INFO:teuthology.orchestra.console:Power on for smithi052 completed 2022-02-17T14:09:54.389 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2022-02-17T14:09:54.399 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish 2022-02-17T14:12:10.586 ERROR:teuthology.orchestra.connection:Error authenticating with smithi052.front.sepia.ceph.com: Authentication failed. 2022-02-17T14:13:42.375 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-17T14:13:57.700 INFO:teuthology.provision.fog.smithi052:Node is ready 2022-02-17T14:13:57.722 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052.front.sepia.ceph.com 2022-02-17T14:13:57.782 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52 smithi052.front.sepia.ceph.com smithi052 2022-02-17T14:13:58.160 INFO:teuthology.provision.fog.smithi052:Deploy complete! 2022-02-17T14:14:41.229 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-17T14:14:41.503 INFO:teuthology.provision.fog.smithi138:Node is ready 2022-02-17T14:14:41.520 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2022-02-17T14:14:41.577 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2022-02-17T14:14:41.915 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2022-02-17T14:14:42.069 INFO:teuthology.lock.ops:Checking smithi052.front.sepia.ceph.com 2022-02-17T14:14:42.071 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-17T14:14:42.091 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2022-02-17T14:14:42.092 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-17T14:14:42.110 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: set os type and version on server 2022-02-17T14:14:42.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2022-02-17T14:14:42.362 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2022-02-17T14:14:42.419 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2022-02-17T14:14:42.420 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="8" 2022-02-17T14:14:42.420 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2022-02-17T14:14:42.420 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2022-02-17T14:14:42.420 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="8" 2022-02-17T14:14:42.421 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el8" 2022-02-17T14:14:42.421 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-17T14:14:42.421 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2022-02-17T14:14:42.421 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-17T14:14:42.422 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2022-02-17T14:14:42.422 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-17T14:14:42.422 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-17T14:14:42.422 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-17T14:14:42.424 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2022-02-17T14:14:42.442 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2022-02-17T14:14:42.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-02-17T14:14:42.631 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2022-02-17T14:14:42.687 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2022-02-17T14:14:42.687 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8" 2022-02-17T14:14:42.688 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2022-02-17T14:14:42.688 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2022-02-17T14:14:42.688 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2022-02-17T14:14:42.689 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2022-02-17T14:14:42.689 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-17T14:14:42.689 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2022-02-17T14:14:42.689 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-17T14:14:42.690 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2022-02-17T14:14:42.690 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-17T14:14:42.690 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-17T14:14:42.690 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-17T14:14:42.691 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-02-17T14:14:42.789 INFO:teuthology.dispatcher.supervisor:Running job 6690268 2022-02-17T14:14:42.789 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690268 --name nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi --description rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} -- /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690268/orig.config.yaml 2022-02-17T14:14:42.793 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690268 2022-02-17T14:14:42.794 INFO:teuthology.dispatcher.supervisor:Job PID: 17193 2022-02-17T14:14:42.794 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-17T20:44:54.406 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-17T20:44:54.612 INFO:teuthology.kill:Killing Pids: {17193} 2022-02-17T20:44:55.679 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-17T20:44:55.680 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-17T20:44:55.680 INFO:teuthology.misc:Compressing logs... 2022-02-17T20:44:55.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2022-02-17T20:44:55.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-02-17T20:44:56.187 INFO:teuthology.misc:Compressing logs... 2022-02-17T20:44:56.756 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-02-17T20:44:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-02-17T20:44:57.251 INFO:teuthology.misc:Compressing logs... 2022-02-17T20:45:01.205 INFO:teuthology.kill:No teuthology processes running 2022-02-17T20:45:02.297 INFO:teuthology.kill:Nuking machines: ['smithi052', 'smithi138'] 2022-02-17T20:45:03.208 INFO:teuthology.kill:2022-02-17 20:45:03,207.207 INFO:teuthology.nuke:targets: 2022-02-17T20:45:03.209 INFO:teuthology.kill: smithi052.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3jxQGhTsKOEdSWfekK9LLTnZ9GC1P5zepSDg8rR2iTnX46RBLve4EP0demnIlx3mHxgmaaHhPreoyGiFwW6sZoY/nkZMJCi6nsyXfBpWhGBMBSoPCrknjuYLhCbpj8VoPSSsjDhmAGEFjdAWERHUCueBoJOQyPbdvABp0eM7TALXiHjAfGXuEXvzHsrpSt7fKyMVEcBdHqZWFcWsyh80Ddh1W6HxiOyc4BMtFaeEdGslF3xsDq9K+OHi7xuohiS/q8KW2T4ACe2i4IfJBocAxEPp/sTKH1CxdUt4KeTmtL/vQv0OX8lntYCrxBdkX8O9oaENN8PfLlA+URACJbaNesA31KYOxFJA1Q9GCDih4fXVRWb1Zrwtv1VlUSMaOcRxgydR/3RJKxpF9UrJ+wwWj/r8LivFJuuUG8igppHEKsbQ9uYsGducKd3qSPOXrkuHQ9xTAO6mHqJPJ6Hiqv0PQjgajnFsW7GMsBeSvljmwQVytQC0vat/L00o+m9Yf4j0= 2022-02-17T20:45:03.209 INFO:teuthology.kill: smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKWf1wGR7nRBth6eW6P90eDxO+7IzKSDs8TaTFgdhDQxiMK7uy9DrXrG+2excmdUSK/VTAUac1gEATw+BarA1efjzIR0CUmR9PSZ0/C27egitlY0JYfi9dgQiGaMmngjeFdgyn0IrzzJMQm2lhC0hJ3MZ5wIQy3vLfrPZoa8vaWHFeBmqaQmogRSexUFLwX2mMTF7V9/h6kkbtbiliZrkl/lyNFzgU0pZoQW4kjX+Cxmukg6h1ciIxapHYnSkdyWz4AuQMhq4fRP7ZtaerMwFrjTNd2NA4inhpQFeIXLpCQ9GJVJky+r1vau/hKWCTogNyV0sflAjXb6PsLrBGvsfKjgZJx+PGMIKqNC0lEiRkH0KxJ/57mRhH0I3B92h0b0LtapLj3UiwyX7/KDDxYy07X0YDYMML/jLtPLJ03kYk+mmKrVnbwXzyJhgFdrI1pS6au2qDf/NS33TCz6H7aV1FJoPF3lmCk3OEakJDsra8G/L3P1oi1dPFLZdCb7BiGCc= 2022-02-17T20:45:03.234 INFO:teuthology.kill:2022-02-17 20:45:03,234.234 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-17T20:45:03.237 INFO:teuthology.kill:2022-02-17 20:45:03,237.237 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-17T20:45:03.312 INFO:teuthology.kill:2022-02-17 20:45:03,312.312 INFO:teuthology.orchestra.console:Power off smithi138 2022-02-17T20:45:03.325 INFO:teuthology.kill:2022-02-17 20:45:03,325.325 INFO:teuthology.orchestra.console:Power off smithi052 2022-02-17T20:45:11.681 INFO:teuthology.kill:2022-02-17 20:45:11,681.681 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-02-17T20:45:11.688 INFO:teuthology.kill:2022-02-17 20:45:11,688.688 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2022-02-17T20:45:11.826 INFO:teuthology.kill:2022-02-17 20:45:11,825.825 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com 2022-02-17T20:45:11.829 INFO:teuthology.kill:2022-02-17 20:45:11,829.829 INFO:teuthology.lock.ops:unlocked: smithi052.front.sepia.ceph.com 2022-02-17T20:47:12.012 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15