2022-02-17T00:16:57.819 INFO:root:teuthology version: 1.1.0-c6442aa9
2022-02-17T00:16:57.944 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi111.front.sepia.ceph.com' reimaging
2022-02-17T00:16:57.945 INFO:teuthology.lock.ops:Updating [ubuntu@smithi111.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T00:16:57.946 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server
2022-02-17T00:16:57.964 INFO:teuthology.lock.ops:Node 'ubuntu@smithi111.front.sepia.ceph.com' reimaging is complete
2022-02-17T00:16:57.965 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging
2022-02-17T00:16:57.965 INFO:teuthology.lock.ops:Updating [ubuntu@smithi162.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T00:16:57.965 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server
2022-02-17T00:16:57.982 INFO:teuthology.lock.ops:Node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging is complete
2022-02-17T00:16:58.177 INFO:teuthology.provision.fog.smithi111:Scheduling deploy of centos 8.stream
2022-02-17T00:16:58.186 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of centos 8.stream
2022-02-17T00:16:58.514 INFO:teuthology.orchestra.console:Power off smithi162
2022-02-17T00:16:58.527 INFO:teuthology.orchestra.console:Power off smithi111
2022-02-17T00:17:10.992 INFO:teuthology.orchestra.console:Power off for smithi162 completed
2022-02-17T00:17:10.997 INFO:teuthology.orchestra.console:Power off for smithi111 completed
2022-02-17T00:17:11.094 INFO:teuthology.orchestra.console:Power on smithi162
2022-02-17T00:17:11.101 INFO:teuthology.orchestra.console:Power on smithi111
2022-02-17T00:17:15.346 INFO:teuthology.orchestra.console:Power on for smithi162 completed
2022-02-17T00:17:15.448 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish
2022-02-17T00:17:19.468 INFO:teuthology.orchestra.console:Power on for smithi111 completed
2022-02-17T00:17:19.569 INFO:teuthology.provision.fog.smithi111:Waiting for deploy to finish
2022-02-17T00:20:05.589 ERROR:teuthology.orchestra.connection:Error authenticating with smithi111.front.sepia.ceph.com: Authentication failed.
2022-02-17T00:21:45.563 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T00:22:02.472 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T00:22:10.818 INFO:teuthology.provision.fog.smithi162:Node is ready
2022-02-17T00:22:10.843 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com
2022-02-17T00:22:10.903 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162
2022-02-17T00:22:11.271 INFO:teuthology.provision.fog.smithi162:Deploy complete!
2022-02-17T00:22:27.784 INFO:teuthology.provision.fog.smithi111:Node is ready
2022-02-17T00:22:27.807 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111.front.sepia.ceph.com
2022-02-17T00:22:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111 smithi111.front.sepia.ceph.com smithi111
2022-02-17T00:22:28.227 INFO:teuthology.provision.fog.smithi111:Deploy complete!
2022-02-17T00:22:28.409 INFO:teuthology.lock.ops:Checking smithi111.front.sepia.ceph.com
2022-02-17T00:22:28.410 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T00:22:28.429 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com
2022-02-17T00:22:28.429 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T00:22:28.447 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: set os type and version on server
2022-02-17T00:22:28.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com'
2022-02-17T00:22:28.636 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64
2022-02-17T00:22:28.693 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream"
2022-02-17T00:22:28.693 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="8"
2022-02-17T00:22:28.693 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos"
2022-02-17T00:22:28.694 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora"
2022-02-17T00:22:28.694 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="8"
2022-02-17T00:22:28.694 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el8"
2022-02-17T00:22:28.694 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-17T00:22:28.695 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31"
2022-02-17T00:22:28.695 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-17T00:22:28.696 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/"
2022-02-17T00:22:28.696 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-17T00:22:28.697 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-17T00:22:28.697 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-17T00:22:28.698 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server
2022-02-17T00:22:28.716 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server
2022-02-17T00:22:28.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com'
2022-02-17T00:22:28.966 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64
2022-02-17T00:22:29.023 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream"
2022-02-17T00:22:29.024 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="8"
2022-02-17T00:22:29.024 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos"
2022-02-17T00:22:29.024 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora"
2022-02-17T00:22:29.024 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="8"
2022-02-17T00:22:29.025 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el8"
2022-02-17T00:22:29.025 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-17T00:22:29.025 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31"
2022-02-17T00:22:29.025 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-17T00:22:29.026 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/"
2022-02-17T00:22:29.026 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-17T00:22:29.026 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-17T00:22:29.026 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-17T00:22:29.027 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server
2022-02-17T00:22:29.272 INFO:teuthology.dispatcher.supervisor:Running job 6690121
2022-02-17T00:22:29.273 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690121 --name nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi --description rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects} -- /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690121/orig.config.yaml
2022-02-17T00:22:29.276 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690121
2022-02-17T00:22:29.277 INFO:teuthology.dispatcher.supervisor:Job PID: 30307
2022-02-17T00:22:29.277 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-17T06:52:39.954 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-17T06:52:40.157 INFO:teuthology.kill:Killing Pids: {30307}
2022-02-17T06:52:41.407 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2022-02-17T06:52:41.408 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2022-02-17T06:52:41.409 INFO:teuthology.misc:Compressing logs...
2022-02-17T06:52:41.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com'
2022-02-17T06:52:41.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com'
2022-02-17T06:52:41.908 INFO:teuthology.misc:Compressing logs...
2022-02-17T06:52:42.476 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-17T06:52:42.589 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-17T06:52:42.969 INFO:teuthology.misc:Compressing logs...
2022-02-17T06:52:47.886 INFO:teuthology.kill:No teuthology processes running
2022-02-17T06:52:49.099 INFO:teuthology.kill:Nuking machines: ['smithi111', 'smithi162']
2022-02-17T06:52:50.083 INFO:teuthology.kill:2022-02-17 06:52:50,083.083 INFO:teuthology.nuke:targets:
2022-02-17T06:52:50.084 INFO:teuthology.kill:  smithi111.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzDddIe7O8GQ6AP4LWJbeCjP3FYmCU4WAsSCx2eYe7vIvMXn9UFSmOa2TM7niNe4eMUhlGRQfZ/szoxS8iyisy7w3IAR0+V7nVf+DYXBa/8rNLRxZGQoIch3yj0pay5CaTEAekONz3o9WUQw0D+HrJb237c+zhkmcQzPb5Qlw+ndK2X1iqpEFF9T/WjraOdHeV9bwOoOV4RBELY//bnrGOphpUYuKT12HhprCDvaT0rdT8BlK1x7K3yWChYCz5F0f9S2ATQgutRzVxLLz2ZQC/r7Ue2IC7reqLJx1hjvyrqkgl87lyI8oDO1D/BaWYgFM6UDssIRnh3yGa/mlJ4z1hbG7yyyZB6bqUJnXzypnZ/Co8zkBi2jJgtgg6EnNHf+5y+SoeAZ26Qx5Fm8qllxgqX+VVYJaX0PfI17q14X5EnB8gH6EbrhTgDTdAIL49pEgjFAr9vuG7u/EVLC1zi8Vi9kPLFmT28fPdwLp1a6d8m4Cpar5FsUjPXfycX8UWi2U=
2022-02-17T06:52:50.084 INFO:teuthology.kill:  smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYAAwmJxPqgrGLnvYTFw3ZXliTbAHim3F6KkJI9uomPHWiRvAh5A80EAvEG0xShacx5NoDtqGO1yPFUT61jH7Yg4jhiN7h9Z9vZACtomS20cvazeE+5ThewuQbt//AZ/GKbWsoSUvKyttDr1toAiNlRZJ97g3HdSmNsVaV1+Ux5aVeiSWtD+HNJB/mPj+fJck+RQcd5yQXcphR3wUjN+pL92yZE9h1oKf/+3gqR+IYdpgEbZlC3Cx2EGzVEprrEUDpwfqDKLjg6tTt6wfOjJSqZ097yP+rWruaFblwl4bvvK14IhzdQgJBXAyt0Tg2jXf1Y0kpG8dSiXWCliFf/D6CQBM4DEOs8n4b+Okri95KvBhnWVA6pqEcuroH9ZI9lTntRVfOJOBbHJA6hELlgU5HEu64knNjJwL5huazMlTWZFFgnm+KOTxy5UJBBN1H9/CzqcWFqZ+/AeuB/Bf8ul6yh6uK93RN0Ufb1lRlpbgZeKtlULnv8oNLc71DzzoDKPs=
2022-02-17T06:52:50.109 INFO:teuthology.kill:2022-02-17 06:52:50,109.109 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T06:52:50.114 INFO:teuthology.kill:2022-02-17 06:52:50,113.113 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T06:52:50.193 INFO:teuthology.kill:2022-02-17 06:52:50,192.192 INFO:teuthology.orchestra.console:Power off smithi111
2022-02-17T06:52:50.208 INFO:teuthology.kill:2022-02-17 06:52:50,208.208 INFO:teuthology.orchestra.console:Power off smithi162
2022-02-17T06:52:58.576 INFO:teuthology.kill:2022-02-17 06:52:58,576.576 INFO:teuthology.orchestra.console:Power off for smithi162 completed
2022-02-17T06:52:58.724 INFO:teuthology.kill:2022-02-17 06:52:58,723.723 INFO:teuthology.lock.ops:unlocked: smithi162.front.sepia.ceph.com
2022-02-17T06:53:02.686 INFO:teuthology.kill:2022-02-17 06:53:02,686.686 INFO:teuthology.orchestra.console:Power off for smithi111 completed
2022-02-17T06:53:02.845 INFO:teuthology.kill:2022-02-17 06:53:02,844.844 INFO:teuthology.lock.ops:unlocked: smithi111.front.sepia.ceph.com
2022-02-17T06:55:03.072 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-17T06:55:03.173 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi111 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690446
2022-02-17T06:55:03.173 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi162 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690446