2022-02-17T07:32:45.050 INFO:root:teuthology version: 1.1.0-c6442aa9
2022-02-17T07:32:45.175 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi090.front.sepia.ceph.com' reimaging
2022-02-17T07:32:45.177 INFO:teuthology.lock.ops:Updating [ubuntu@smithi090.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T07:32:45.178 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2022-02-17T07:32:45.198 INFO:teuthology.lock.ops:Node 'ubuntu@smithi090.front.sepia.ceph.com' reimaging is complete
2022-02-17T07:32:45.198 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging
2022-02-17T07:32:45.199 INFO:teuthology.lock.ops:Updating [ubuntu@smithi120.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T07:32:45.199 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2022-02-17T07:32:45.225 INFO:teuthology.lock.ops:Node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging is complete
2022-02-17T07:32:45.409 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of rhel 8.4
2022-02-17T07:32:45.427 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.4
2022-02-17T07:32:45.791 INFO:teuthology.orchestra.console:Power off smithi120
2022-02-17T07:32:45.807 INFO:teuthology.orchestra.console:Power off smithi090
2022-02-17T07:32:45.936 INFO:teuthology.orchestra.console:Power off for smithi120 completed
2022-02-17T07:32:45.949 INFO:teuthology.orchestra.console:Power off for smithi090 completed
2022-02-17T07:32:46.039 INFO:teuthology.orchestra.console:Power on smithi120
2022-02-17T07:32:46.050 INFO:teuthology.orchestra.console:Power on smithi090
2022-02-17T07:32:50.292 INFO:teuthology.orchestra.console:Power on for smithi120 completed
2022-02-17T07:32:50.297 INFO:teuthology.orchestra.console:Power on for smithi090 completed
2022-02-17T07:32:50.394 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish
2022-02-17T07:32:50.398 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish
2022-02-17T07:37:31.440 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T07:37:31.679 INFO:teuthology.provision.fog.smithi120:Node is ready
2022-02-17T07:37:31.694 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com
2022-02-17T07:37:31.750 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120
2022-02-17T07:37:32.069 INFO:teuthology.provision.fog.smithi120:Deploy complete!
2022-02-17T07:38:10.750 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T07:38:11.019 INFO:teuthology.provision.fog.smithi090:Node is ready
2022-02-17T07:38:11.035 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090.front.sepia.ceph.com
2022-02-17T07:38:11.091 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90 smithi090.front.sepia.ceph.com smithi090
2022-02-17T07:38:11.424 INFO:teuthology.provision.fog.smithi090:Deploy complete!
2022-02-17T07:38:11.575 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com
2022-02-17T07:38:11.576 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T07:38:11.596 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com
2022-02-17T07:38:11.596 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T07:38:11.619 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server
2022-02-17T07:38:11.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2022-02-17T07:38:11.838 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64
2022-02-17T07:38:11.892 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Red Hat Enterprise Linux"
2022-02-17T07:38:11.893 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="8.4 (Ootpa)"
2022-02-17T07:38:11.893 INFO:teuthology.orchestra.run.smithi090.stdout:ID="rhel"
2022-02-17T07:38:11.893 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="fedora"
2022-02-17T07:38:11.893 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="8.4"
2022-02-17T07:38:11.894 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el8"
2022-02-17T07:38:11.894 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-02-17T07:38:11.894 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2022-02-17T07:38:11.894 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-02-17T07:38:11.895 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.redhat.com/"
2022-02-17T07:38:11.895 INFO:teuthology.orchestra.run.smithi090.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-02-17T07:38:11.895 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-17T07:38:11.895 INFO:teuthology.orchestra.run.smithi090.stdout:
2022-02-17T07:38:11.896 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-17T07:38:11.896 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-02-17T07:38:11.896 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-02-17T07:38:11.896 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-02-17T07:38:11.898 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2022-02-17T07:38:11.943 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server
2022-02-17T07:38:11.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2022-02-17T07:38:12.178 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64
2022-02-17T07:38:12.234 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux"
2022-02-17T07:38:12.234 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.4 (Ootpa)"
2022-02-17T07:38:12.234 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel"
2022-02-17T07:38:12.235 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora"
2022-02-17T07:38:12.235 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.4"
2022-02-17T07:38:12.238 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8"
2022-02-17T07:38:12.238 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-02-17T07:38:12.238 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31"
2022-02-17T07:38:12.239 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-02-17T07:38:12.239 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/"
2022-02-17T07:38:12.239 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-02-17T07:38:12.239 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-17T07:38:12.239 INFO:teuthology.orchestra.run.smithi120.stdout:
2022-02-17T07:38:12.240 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-17T07:38:12.240 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-02-17T07:38:12.240 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-02-17T07:38:12.240 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-02-17T07:38:12.241 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2022-02-17T07:38:12.337 INFO:teuthology.dispatcher.supervisor:Running job 6690465
2022-02-17T07:38:12.338 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690465 --name benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} -- /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690465/orig.config.yaml
2022-02-17T07:38:12.341 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690465
2022-02-17T07:38:12.343 INFO:teuthology.dispatcher.supervisor:Job PID: 25322
2022-02-17T07:38:12.343 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-17T14:08:41.955 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-17T14:08:42.157 INFO:teuthology.kill:Killing Pids: {25322}
2022-02-17T14:08:43.373 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2022-02-17T14:08:43.373 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2022-02-17T14:08:43.374 INFO:teuthology.misc:Compressing logs...
2022-02-17T14:08:43.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2022-02-17T14:08:43.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2022-02-17T14:08:43.907 INFO:teuthology.misc:Compressing logs...
2022-02-17T14:08:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-17T14:08:44.648 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-17T14:08:45.029 INFO:teuthology.misc:Compressing logs...
2022-02-17T14:08:46.453 INFO:teuthology.kill:No teuthology processes running
2022-02-17T14:08:47.660 INFO:teuthology.kill:Nuking machines: ['smithi090', 'smithi120']
2022-02-17T14:08:48.638 INFO:teuthology.kill:2022-02-17 14:08:48,638.638 INFO:teuthology.nuke:targets:
2022-02-17T14:08:48.639 INFO:teuthology.kill:  smithi090.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCaMo/YuxyGtmDN4VJrIHNr83RGKwDlrXj3F7Uh8uK3Qf6V2Dya9TCWgWtsQzR58LmS9TAc9F29H8Z2yoPnRSank5DhNXMUj9bwnbmm6UyHX/i/IG3WPvUWjaQAJPm0VQ8MXkQ5UojOnkYkYccV/uD3NfzCFKe0ALDrfa85ub1aoQzWRR0GZ0hNHbsMFuY/D9SDqTL63CQ0uQZ2BBBbkJ/BnaCrGoepBqU5Q3umtNVzVUfZNS3beSdCjF8ByZpfg6YqYguuhHBXyafRy26X8QbjZX3xJcH6D0j9N7TB8AoIgegQm/2m+XpLYfAshjCa/kMuhcA67pAPS8hRwNoQEGjlxu13LESrxWhzN0ApfIlwT7lPVnfwmbAx2u7BumfKayhmLk0D/Owuee2qd+pT9HRiVyO56W8uvKfXIBqPFsUE9EkMsNfed4ozKNoByfTQfp0/MlGiO3GGYBKoOOeAaOozkLDIcWA3cG4v/ISBTojDFqdeAiEiGS8KZ1A3U/ZljM=
2022-02-17T14:08:48.639 INFO:teuthology.kill:  smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8Wa2zQwr1m3WFEtykFGDupKko6aoDtrfQr76GH0vqfLnEHWkEsfZUgJqocv9+BDdYBjMMo0hyaH4qedGgvc82FgUIDJf6MbWwyOM+y2vPPMDT8pDxfVQqrrYo8javi4Jvvj0csIJUky+J9UU7ZmA+0AMrBLdWGxvTGU/KiccGCmskAUiZvuRm2m6hCi37NsC2fLrGAX4Qag5dsOaMp2RG19me9gg5OVAOIvk9BwaM0qcrf7kpj65KeCrD8X+qLXeou5bmYx0NuqHPBV+hFrSEBmXjUI7d2MzSDsRvvgbM6fZuhv7rE05M15ZqYgu9QyPRfCFERXlNYaPrCHARdUyW0nYrwAw8LOfYie9yReO0rFHlf4o4meYoV/82Jq0R4rEzPKurwc+M8jF2FrKk7hgrchvk/tOvcLc99pp9kDkcLko7ZStslXBqhgkVE134JGyxIIAwldtOecXZAKRyJMDItwLsLiKOTnvxr7agy6GtKwuxBhcC7gN9uG67jH0Zdls=
2022-02-17T14:08:48.667 INFO:teuthology.kill:2022-02-17 14:08:48,667.667 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T14:08:48.672 INFO:teuthology.kill:2022-02-17 14:08:48,672.672 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T14:08:48.749 INFO:teuthology.kill:2022-02-17 14:08:48,748.748 INFO:teuthology.orchestra.console:Power off smithi120
2022-02-17T14:08:48.763 INFO:teuthology.kill:2022-02-17 14:08:48,762.762 INFO:teuthology.orchestra.console:Power off smithi090
2022-02-17T14:08:57.124 INFO:teuthology.kill:2022-02-17 14:08:57,124.124 INFO:teuthology.orchestra.console:Power off for smithi090 completed
2022-02-17T14:08:57.270 INFO:teuthology.kill:2022-02-17 14:08:57,270.270 INFO:teuthology.lock.ops:unlocked: smithi090.front.sepia.ceph.com
2022-02-17T14:09:01.229 INFO:teuthology.kill:2022-02-17 14:09:01,228.228 INFO:teuthology.orchestra.console:Power off for smithi120 completed
2022-02-17T14:09:01.375 INFO:teuthology.kill:2022-02-17 14:09:01,374.374 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com
2022-02-17T14:11:01.689 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-17T14:11:01.792 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi090 but it was locked by another job: /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690267
2022-02-17T14:11:01.793 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi120 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-17_03:05:02-rgw-master-distro-default-smithi/6690360