2022-02-17T05:48:19.935 INFO:root:teuthology version: 1.1.0-c6442aa9
2022-02-17T05:48:20.085 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi058.front.sepia.ceph.com' reimaging
2022-02-17T05:48:20.086 INFO:teuthology.lock.ops:Updating [ubuntu@smithi058.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T05:48:20.086 INFO:teuthology.lock.ops:Updating smithi058.front.sepia.ceph.com on lock server
2022-02-17T05:48:20.140 INFO:teuthology.lock.ops:Node 'ubuntu@smithi058.front.sepia.ceph.com' reimaging is complete
2022-02-17T05:48:20.141 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging
2022-02-17T05:48:20.141 INFO:teuthology.lock.ops:Updating [ubuntu@smithi077.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T05:48:20.142 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server
2022-02-17T05:48:20.160 INFO:teuthology.lock.ops:Node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging is complete
2022-02-17T05:48:20.160 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi105.front.sepia.ceph.com' reimaging
2022-02-17T05:48:20.161 INFO:teuthology.lock.ops:Updating [ubuntu@smithi105.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T05:48:20.161 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2022-02-17T05:48:20.182 INFO:teuthology.lock.ops:Node 'ubuntu@smithi105.front.sepia.ceph.com' reimaging is complete
2022-02-17T05:48:20.355 INFO:teuthology.provision.fog.smithi058:Scheduling deploy of ubuntu 20.04
2022-02-17T05:48:20.380 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of ubuntu 20.04
2022-02-17T05:48:20.398 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of ubuntu 20.04
2022-02-17T05:48:20.656 INFO:teuthology.orchestra.console:Power off smithi058
2022-02-17T05:48:20.713 INFO:teuthology.orchestra.console:Power off smithi077
2022-02-17T05:48:20.725 INFO:teuthology.orchestra.console:Power off smithi105
2022-02-17T05:48:20.800 INFO:teuthology.orchestra.console:Power off for smithi058 completed
2022-02-17T05:48:20.855 INFO:teuthology.orchestra.console:Power off for smithi077 completed
2022-02-17T05:48:20.866 INFO:teuthology.orchestra.console:Power off for smithi105 completed
2022-02-17T05:48:20.901 INFO:teuthology.orchestra.console:Power on smithi058
2022-02-17T05:48:20.955 INFO:teuthology.orchestra.console:Power on smithi077
2022-02-17T05:48:20.967 INFO:teuthology.orchestra.console:Power on smithi105
2022-02-17T05:48:25.158 INFO:teuthology.orchestra.console:Power on for smithi058 completed
2022-02-17T05:48:25.259 INFO:teuthology.provision.fog.smithi058:Waiting for deploy to finish
2022-02-17T05:48:29.327 INFO:teuthology.orchestra.console:Power on for smithi105 completed
2022-02-17T05:48:29.359 INFO:teuthology.orchestra.console:Power on for smithi077 completed
2022-02-17T05:48:29.429 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish
2022-02-17T05:48:29.460 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish
2022-02-17T05:53:02.756 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T05:53:03.291 INFO:teuthology.provision.fog.smithi058:Node is ready
2022-02-17T05:53:03.298 INFO:teuthology.orchestra.run.smithi058.stdout:smithi058.front.sepia.ceph.com
2022-02-17T05:53:03.349 INFO:teuthology.orchestra.run.smithi058.stdout:172.21.15.58 smithi058.front.sepia.ceph.com smithi058
2022-02-17T05:53:03.518 INFO:teuthology.provision.fog.smithi058:Deploy complete!
2022-02-17T05:53:26.437 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T05:53:26.861 INFO:teuthology.provision.fog.smithi105:Node is ready
2022-02-17T05:53:26.867 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105.front.sepia.ceph.com
2022-02-17T05:53:26.875 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105 smithi105.front.sepia.ceph.com smithi105
2022-02-17T05:53:27.039 INFO:teuthology.provision.fog.smithi105:Deploy complete!
2022-02-17T05:53:27.543 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T05:53:28.016 INFO:teuthology.provision.fog.smithi077:Node is ready
2022-02-17T05:53:28.065 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com
2022-02-17T05:53:28.072 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077
2022-02-17T05:53:28.183 INFO:teuthology.provision.fog.smithi077:Deploy complete!
2022-02-17T05:53:28.416 INFO:teuthology.lock.ops:Checking smithi058.front.sepia.ceph.com
2022-02-17T05:53:28.417 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T05:53:28.436 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com
2022-02-17T05:53:28.436 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T05:53:28.453 INFO:teuthology.lock.ops:Checking smithi105.front.sepia.ceph.com
2022-02-17T05:53:28.454 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T05:53:28.474 INFO:teuthology.lock.ops:Updating [smithi058.front.sepia.ceph.com]: set os type and version on server
2022-02-17T05:53:28.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi058.front.sepia.ceph.com'
2022-02-17T05:53:28.731 INFO:teuthology.orchestra.run.smithi058.stdout:x86_64
2022-02-17T05:53:28.736 INFO:teuthology.orchestra.run.smithi058.stdout:NAME="Ubuntu"
2022-02-17T05:53:28.737 INFO:teuthology.orchestra.run.smithi058.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T05:53:28.737 INFO:teuthology.orchestra.run.smithi058.stdout:ID=ubuntu
2022-02-17T05:53:28.737 INFO:teuthology.orchestra.run.smithi058.stdout:ID_LIKE=debian
2022-02-17T05:53:28.737 INFO:teuthology.orchestra.run.smithi058.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T05:53:28.738 INFO:teuthology.orchestra.run.smithi058.stdout:VERSION_ID="20.04"
2022-02-17T05:53:28.738 INFO:teuthology.orchestra.run.smithi058.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T05:53:28.738 INFO:teuthology.orchestra.run.smithi058.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T05:53:28.738 INFO:teuthology.orchestra.run.smithi058.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T05:53:28.739 INFO:teuthology.orchestra.run.smithi058.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T05:53:28.739 INFO:teuthology.orchestra.run.smithi058.stdout:VERSION_CODENAME=focal
2022-02-17T05:53:28.739 INFO:teuthology.orchestra.run.smithi058.stdout:UBUNTU_CODENAME=focal
2022-02-17T05:53:28.740 INFO:teuthology.lock.ops:Updating smithi058.front.sepia.ceph.com on lock server
2022-02-17T05:53:28.763 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server
2022-02-17T05:53:28.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com'
2022-02-17T05:53:28.973 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64
2022-02-17T05:53:28.979 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="Ubuntu"
2022-02-17T05:53:28.980 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T05:53:28.980 INFO:teuthology.orchestra.run.smithi077.stdout:ID=ubuntu
2022-02-17T05:53:28.980 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE=debian
2022-02-17T05:53:28.980 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T05:53:28.981 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="20.04"
2022-02-17T05:53:28.981 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T05:53:28.981 INFO:teuthology.orchestra.run.smithi077.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T05:53:28.981 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T05:53:28.982 INFO:teuthology.orchestra.run.smithi077.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T05:53:28.982 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_CODENAME=focal
2022-02-17T05:53:28.982 INFO:teuthology.orchestra.run.smithi077.stdout:UBUNTU_CODENAME=focal
2022-02-17T05:53:28.982 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server
2022-02-17T05:53:29.002 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: set os type and version on server
2022-02-17T05:53:29.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com'
2022-02-17T05:53:29.178 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64
2022-02-17T05:53:29.185 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Ubuntu"
2022-02-17T05:53:29.185 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T05:53:29.186 INFO:teuthology.orchestra.run.smithi105.stdout:ID=ubuntu
2022-02-17T05:53:29.186 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE=debian
2022-02-17T05:53:29.186 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T05:53:29.187 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="20.04"
2022-02-17T05:53:29.187 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T05:53:29.187 INFO:teuthology.orchestra.run.smithi105.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T05:53:29.187 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T05:53:29.187 INFO:teuthology.orchestra.run.smithi105.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T05:53:29.188 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_CODENAME=focal
2022-02-17T05:53:29.188 INFO:teuthology.orchestra.run.smithi105.stdout:UBUNTU_CODENAME=focal
2022-02-17T05:53:29.188 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2022-02-17T05:53:29.283 INFO:teuthology.dispatcher.supervisor:Running job 6690431
2022-02-17T05:53:29.284 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690431 --name benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690431/orig.config.yaml
2022-02-17T05:53:29.288 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690431
2022-02-17T05:53:29.289 INFO:teuthology.dispatcher.supervisor:Job PID: 7068
2022-02-17T05:53:29.289 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-17T12:23:59.447 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-17T12:23:59.696 INFO:teuthology.kill:Killing Pids: {7068}
2022-02-17T12:24:00.941 INFO:teuthology.task.internal:roles: ubuntu@smithi058.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-02-17T12:24:00.942 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-02-17T12:24:00.942 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-02-17T12:24:00.942 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:24:00.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi058.front.sepia.ceph.com'
2022-02-17T12:24:01.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com'
2022-02-17T12:24:01.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com'
2022-02-17T12:24:01.573 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:24:02.297 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:24:14.236 INFO:teuthology.kill:No teuthology processes running
2022-02-17T12:24:15.521 INFO:teuthology.kill:Nuking machines: ['smithi058', 'smithi077', 'smithi105']
2022-02-17T12:24:16.515 INFO:teuthology.kill:2022-02-17 12:24:16,514.514 INFO:teuthology.nuke:targets:
2022-02-17T12:24:16.516 INFO:teuthology.kill:  smithi058.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEEN9qc0KwzglcNoJ6NULxmN519/y7jpEGE+7aqYdjdZz00X5D9qVhLWmO4TGvutWTL4zxLUJDYJ5jNYIRMb6MuYqvD1if5LbV2nJ8+byfEuTHVrMkAntfvQo3Cm4vLF4uRxmy71/6jskB/u/2UsvKCVsjCDlVfxDAGA5NMAgTHItCk5kHibPIesvBbKWNkoUTe/XURtx+EQzB2Yik09GkHtATJnczICoEVqV15HAiRjnqC9LrJDFucDg0In5+oikitaUdrbuTiut2gYiLfqkKWiQt99BL62RRehwtsgJ9siSeoMpxfOpEC5XYvr0mp/0ryENuxaF/LeuQwBBugnl4aXzNN0RfxmYEQ2McKKiC/4GeqmQsKvjP0iUtqsN8kpiQ3JdIpmFg6S4d1h0j7n1vhChIGTNFMYo0NrJ5pHmXvoEfmdDfMXRMLdp1eUyT4guV9tC5pxKzL3qehHubhxWscJ/uAHY7jkd8sw50Z/TokFcbo7ZGmwtNlpLVbzv0nhM=
2022-02-17T12:24:16.516 INFO:teuthology.kill:  smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjptzf9jj0m1yj7qmNpxxOmjzIUcx55gD5n0ztBHdpjqQ9gmGFdRkXrDgvBPZwe/vpj60huNRWn1Vn5DOPE6RwXYg135612R5NEjOg0kJ0wWqNRGohIttfN4NEqecuRVe1n8sxwnx0422Qc8jz9UZ2IFggXQ6dpEH7WrlRlO/FUx5ik2IQ7Pajs8M4I1zNNz3gEU5wmBcCrp4LRnZLn/Rhylhx+w7h3Z2IdZCBqJo42csUOMPIAWv426Msz7FQ1o80sa2Vh41zVo7kkmyswgAXxT3hgGk1k5cd9FMYvaftlyiRRzzzhNmrfpgIOB5cF8ObnUbbeujmT0hKnV4SW6pVwOYCf626YXxKALXvxeslyDj8E1rX4oMQ8qb2zzMxTE26VlNDD3DEsblMHlvZCuv6TQYtDw9dZ0GrE5L6bcdnAqxcLoTSqx/LjTLg0oE6DFSfHBW7ILPIFEM8H5027VVKSMDgj6ww/xak4iQTsGEPm7XC5scmB2iRXY9CZu/GGCs=
2022-02-17T12:24:16.516 INFO:teuthology.kill:  smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4DFyJUaFuTcA15aiKexEjxVf4AAL8Bi9vBL3wAYqet+V4EKzkM42GqI7y3tH1vedkxTPY5OuqtnVMVYPQDAC/K/pjmFKMwQGUH0QpEodEiriLtaoNti0VCiCMAcQeiko5/RYCiEW7+vYTLdhSY3oNl4FWvTgzpbuhcvRHDA8pLdRn0lMghJF8SuLK+DREWorm4NvSbHm/K+cA7qHEzhB8CyMPon3f+qM4z9YQ4m/5bXdg3mH0ReZQ6iJzMFbB+tRqZQ5WiOcbodRC08oR4zRiAYVQWmrqOVRzZab3ABlMp8QPBGk2JWvIoRazwnIBg8LlhYuDeLmRKyrPakffN6sDZ5/RDapcolQYN2ucL9T5nA0iXGCqLKH/bhZU/mMqGzrnp91YPv6MedzoEGBEqBLfDVzd/hRgxeP1YxBELwuzouy7dIlwGb+3r/Lm8qsxogJxy6OICtSzzOLgWBpcDbsAJk1v5V6Q+pTYJ16lr369AnZLRYgIKYhPXHPsUL95D8k=
2022-02-17T12:24:16.545 INFO:teuthology.kill:2022-02-17 12:24:16,545.545 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:24:16.549 INFO:teuthology.kill:2022-02-17 12:24:16,549.549 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:24:16.550 INFO:teuthology.kill:2022-02-17 12:24:16,550.550 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:24:16.633 INFO:teuthology.kill:2022-02-17 12:24:16,633.633 INFO:teuthology.orchestra.console:Power off smithi105
2022-02-17T12:24:16.648 INFO:teuthology.kill:2022-02-17 12:24:16,647.647 INFO:teuthology.orchestra.console:Power off smithi077
2022-02-17T12:24:16.659 INFO:teuthology.kill:2022-02-17 12:24:16,658.658 INFO:teuthology.orchestra.console:Power off smithi058
2022-02-17T12:24:25.023 INFO:teuthology.kill:2022-02-17 12:24:25,023.023 INFO:teuthology.orchestra.console:Power off for smithi105 completed
2022-02-17T12:24:25.029 INFO:teuthology.kill:2022-02-17 12:24:25,029.029 INFO:teuthology.orchestra.console:Power off for smithi058 completed
2022-02-17T12:24:25.175 INFO:teuthology.kill:2022-02-17 12:24:25,174.174 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com
2022-02-17T12:24:25.178 INFO:teuthology.kill:2022-02-17 12:24:25,178.178 INFO:teuthology.lock.ops:unlocked: smithi058.front.sepia.ceph.com
2022-02-17T12:24:29.128 INFO:teuthology.kill:2022-02-17 12:24:29,128.128 INFO:teuthology.orchestra.console:Power off for smithi077 completed
2022-02-17T12:24:29.284 INFO:teuthology.kill:2022-02-17 12:24:29,283.283 INFO:teuthology.lock.ops:unlocked: smithi077.front.sepia.ceph.com
2022-02-17T12:26:29.602 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-17T12:26:29.724 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi058 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-17_12:11:23-rados-safe_shutdown_v2_compact_log_async-distro-basic-smithi/6690983
2022-02-17T12:26:29.724 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi077 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-17_12:11:23-rados-safe_shutdown_v2_compact_log_async-distro-basic-smithi/6690983