2022-02-17T00:02:20.716 INFO:root:teuthology version: 1.1.0-c6442aa9
2022-02-17T00:02:20.863 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi037.front.sepia.ceph.com' reimaging
2022-02-17T00:02:20.864 INFO:teuthology.lock.ops:Updating [ubuntu@smithi037.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T00:02:20.864 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server
2022-02-17T00:02:20.882 INFO:teuthology.lock.ops:Node 'ubuntu@smithi037.front.sepia.ceph.com' reimaging is complete
2022-02-17T00:02:20.882 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi050.front.sepia.ceph.com' reimaging
2022-02-17T00:02:20.883 INFO:teuthology.lock.ops:Updating [ubuntu@smithi050.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T00:02:20.883 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2022-02-17T00:02:20.904 INFO:teuthology.lock.ops:Node 'ubuntu@smithi050.front.sepia.ceph.com' reimaging is complete
2022-02-17T00:02:21.110 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of centos 8.stream
2022-02-17T00:02:21.130 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 8.stream
2022-02-17T00:02:21.421 INFO:teuthology.orchestra.console:Power off smithi037
2022-02-17T00:02:21.434 INFO:teuthology.orchestra.console:Power off smithi050
2022-02-17T00:02:29.799 INFO:teuthology.orchestra.console:Power off for smithi050 completed
2022-02-17T00:02:29.901 INFO:teuthology.orchestra.console:Power on smithi050
2022-02-17T00:02:33.902 INFO:teuthology.orchestra.console:Power off for smithi037 completed
2022-02-17T00:02:34.004 INFO:teuthology.orchestra.console:Power on smithi037
2022-02-17T00:02:38.271 INFO:teuthology.orchestra.console:Power on for smithi050 completed
2022-02-17T00:02:38.372 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish
2022-02-17T00:02:42.381 INFO:teuthology.orchestra.console:Power on for smithi037 completed
2022-02-17T00:02:42.482 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish
2022-02-17T00:06:46.024 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T00:06:46.290 INFO:teuthology.provision.fog.smithi050:Node is ready
2022-02-17T00:06:46.307 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com
2022-02-17T00:06:46.365 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050
2022-02-17T00:06:46.718 INFO:teuthology.provision.fog.smithi050:Deploy complete!
2022-02-17T00:06:50.061 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T00:06:50.343 INFO:teuthology.provision.fog.smithi037:Node is ready
2022-02-17T00:06:50.362 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com
2022-02-17T00:06:50.420 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037
2022-02-17T00:06:50.769 INFO:teuthology.provision.fog.smithi037:Deploy complete!
2022-02-17T00:06:50.927 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com
2022-02-17T00:06:50.928 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T00:06:50.947 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com
2022-02-17T00:06:50.948 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T00:06:50.966 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server
2022-02-17T00:06:50.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com'
2022-02-17T00:06:51.159 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64
2022-02-17T00:06:51.217 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream"
2022-02-17T00:06:51.217 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8"
2022-02-17T00:06:51.217 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos"
2022-02-17T00:06:51.217 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora"
2022-02-17T00:06:51.218 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8"
2022-02-17T00:06:51.218 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8"
2022-02-17T00:06:51.218 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-17T00:06:51.219 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31"
2022-02-17T00:06:51.219 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-17T00:06:51.219 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/"
2022-02-17T00:06:51.219 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-17T00:06:51.220 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-17T00:06:51.220 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-17T00:06:51.221 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server
2022-02-17T00:06:51.239 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server
2022-02-17T00:06:51.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com'
2022-02-17T00:06:51.440 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64
2022-02-17T00:06:51.500 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream"
2022-02-17T00:06:51.501 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8"
2022-02-17T00:06:51.501 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos"
2022-02-17T00:06:51.501 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora"
2022-02-17T00:06:51.501 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8"
2022-02-17T00:06:51.502 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8"
2022-02-17T00:06:51.502 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-17T00:06:51.502 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31"
2022-02-17T00:06:51.502 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-17T00:06:51.503 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/"
2022-02-17T00:06:51.503 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-17T00:06:51.503 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-17T00:06:51.503 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-17T00:06:51.504 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2022-02-17T00:06:51.636 INFO:teuthology.dispatcher.supervisor:Running job 6690105
2022-02-17T00:06:51.636 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690105 --name nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi --description rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} -- /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690105/orig.config.yaml
2022-02-17T00:06:51.640 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690105
2022-02-17T00:06:51.640 INFO:teuthology.dispatcher.supervisor:Job PID: 10788
2022-02-17T00:06:51.641 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-17T06:37:20.571 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-17T06:37:20.807 INFO:teuthology.kill:Killing Pids: {10788}
2022-02-17T06:37:22.119 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2022-02-17T06:37:22.120 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2022-02-17T06:37:22.121 INFO:teuthology.misc:Compressing logs...
2022-02-17T06:37:22.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com'
2022-02-17T06:37:22.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com'
2022-02-17T06:37:22.633 INFO:teuthology.misc:Compressing logs...
2022-02-17T06:37:23.198 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-17T06:37:23.380 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-17T06:37:23.765 INFO:teuthology.misc:Compressing logs...
2022-02-17T06:37:45.619 INFO:teuthology.kill:No teuthology processes running
2022-02-17T06:37:46.806 INFO:teuthology.kill:Nuking machines: ['smithi037', 'smithi050']
2022-02-17T06:37:47.837 INFO:teuthology.kill:2022-02-17 06:37:47,837.837 INFO:teuthology.nuke:targets:
2022-02-17T06:37:47.838 INFO:teuthology.kill:  smithi037.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBQWB3rV+OZDPPvNN+SC7bpxDak863TPzGeA1hvQfhJaBgYUi39obtuQK+0O5vynNrh1TAevOV97MgIVMgwFmYiB/hchslENl7PqXjmAQ13+vdHI2j/fQcxo57dTGY+8e5xjXLlnQS1DEJLTxJ8ewFFbQRWbjaoPW4+eao6z+UbZFNm8uDy2tFRp9viU0leANheM0jNbeWJabPqKI9AdG3kA77+Q3HKhXEQFov8O1z08EuEFUULmavwtqkx+VfbKwD065vldKuQWHa8gAM6Z8mW0SkjgPTOW1M12hQpkXv6MWbkO1I19Pz1BnOYVupmGmjpfJTzeZCV8t6yvjeK7m3bQtBh4RqjcA4spAsLVxKhnPQCOVNX42Tkg1vOdbysIJzpvWjGtYUWvcMz3ybevPhPVc+mvVIlPpg+9w8VaFvt7uuE8w2shJUs3iEEhCvoAJz6Z35ikri0eqmZifCLLF7vdLPdYDiVwexVEyW9oSRPVTNu+i2r1on9IIvCfjzP+M=
2022-02-17T06:37:47.838 INFO:teuthology.kill:  smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw7tlJFnYuR+2tjjtsjbueLDA4W3j70q929p/sMlaAUxEuR5oohZa5reMhF5x5dqDoxBbtyqmX77VKrNJEI6OHcjHFoNBj8Oi+Cl20qynOYLGSPkNA/hU87hwJ2rdQmYRPoUDBau699CsPFSiBopQoVMVMVokh5Py/RF6m7CND0IApVDoQ9nfk2vbgmUslrFQw4jmSvNK89TMfH5MPt0yx+UztGRruPocH+NR3Q1HrSm7hSmzfLjf0S3vt2MfSlPC6X1nmkDJJ1UefcMyqjXCyAc9f2lbqVeZA2qTyiqv6HOmEmUThW5R/dQystRNYxcFsCk+K8JBTD+yNyvGhK2c4Klho5/ImSzBfOUT1TQsW6LIw3I5kMuVuBELXfJd4UwQ4C8SKDdbM9NwI15HNghRYhhNCdEGhbtxjRgGllFQ00RvgOWu7MNebKhF6LbXaT2LIZaHlJfbhr2jUYGmrFzO/T3xu0/NH0N4ZDtzXgQzqb+ZPR1x3UrVWID0fLjDu65E=
2022-02-17T06:37:47.865 INFO:teuthology.kill:2022-02-17 06:37:47,865.865 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T06:37:47.866 INFO:teuthology.kill:2022-02-17 06:37:47,866.866 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T06:37:47.949 INFO:teuthology.kill:2022-02-17 06:37:47,949.949 INFO:teuthology.orchestra.console:Power off smithi037
2022-02-17T06:37:47.964 INFO:teuthology.kill:2022-02-17 06:37:47,964.964 INFO:teuthology.orchestra.console:Power off smithi050
2022-02-17T06:37:56.329 INFO:teuthology.kill:2022-02-17 06:37:56,329.329 INFO:teuthology.orchestra.console:Power off for smithi050 completed
2022-02-17T06:37:56.481 INFO:teuthology.kill:2022-02-17 06:37:56,480.480 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com
2022-02-17T06:38:00.432 INFO:teuthology.kill:2022-02-17 06:38:00,431.431 INFO:teuthology.orchestra.console:Power off for smithi037 completed
2022-02-17T06:38:00.579 INFO:teuthology.kill:2022-02-17 06:38:00,579.579 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com
2022-02-17T06:40:00.894 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-17T06:40:01.018 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi037 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690438
2022-02-17T06:40:01.019 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi050 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690438