2022-02-17T05:26:44.050 INFO:root:teuthology version: 1.1.0-c6442aa9
2022-02-17T05:26:44.207 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi042.front.sepia.ceph.com' reimaging
2022-02-17T05:26:44.208 INFO:teuthology.lock.ops:Updating [ubuntu@smithi042.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T05:26:44.209 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server
2022-02-17T05:26:44.228 INFO:teuthology.lock.ops:Node 'ubuntu@smithi042.front.sepia.ceph.com' reimaging is complete
2022-02-17T05:26:44.228 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi045.front.sepia.ceph.com' reimaging
2022-02-17T05:26:44.229 INFO:teuthology.lock.ops:Updating [ubuntu@smithi045.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T05:26:44.229 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server
2022-02-17T05:26:44.248 INFO:teuthology.lock.ops:Node 'ubuntu@smithi045.front.sepia.ceph.com' reimaging is complete
2022-02-17T05:26:44.248 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi142.front.sepia.ceph.com' reimaging
2022-02-17T05:26:44.248 INFO:teuthology.lock.ops:Updating [ubuntu@smithi142.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T05:26:44.249 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server
2022-02-17T05:26:44.267 INFO:teuthology.lock.ops:Node 'ubuntu@smithi142.front.sepia.ceph.com' reimaging is complete
2022-02-17T05:26:44.455 INFO:teuthology.provision.fog.smithi042:Scheduling deploy of ubuntu 20.04
2022-02-17T05:26:44.460 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of ubuntu 20.04
2022-02-17T05:26:44.486 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of ubuntu 20.04
2022-02-17T05:26:44.766 INFO:teuthology.orchestra.console:Power off smithi045
2022-02-17T05:26:44.787 INFO:teuthology.orchestra.console:Power off smithi042
2022-02-17T05:26:44.825 INFO:teuthology.orchestra.console:Power off smithi142
2022-02-17T05:26:53.186 INFO:teuthology.orchestra.console:Power off for smithi142 completed
2022-02-17T05:26:53.288 INFO:teuthology.orchestra.console:Power on smithi142
2022-02-17T05:26:57.278 INFO:teuthology.orchestra.console:Power off for smithi045 completed
2022-02-17T05:26:57.283 INFO:teuthology.orchestra.console:Power off for smithi042 completed
2022-02-17T05:26:57.379 INFO:teuthology.orchestra.console:Power on smithi045
2022-02-17T05:26:57.387 INFO:teuthology.orchestra.console:Power on smithi042
2022-02-17T05:27:01.650 INFO:teuthology.orchestra.console:Power on for smithi142 completed
2022-02-17T05:27:01.751 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish
2022-02-17T05:27:05.750 INFO:teuthology.orchestra.console:Power on for smithi045 completed
2022-02-17T05:27:05.845 INFO:teuthology.orchestra.console:Power on for smithi042 completed
2022-02-17T05:27:05.851 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish
2022-02-17T05:27:05.946 INFO:teuthology.provision.fog.smithi042:Waiting for deploy to finish
2022-02-17T05:31:43.427 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T05:31:43.647 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T05:31:44.001 INFO:teuthology.provision.fog.smithi045:Node is ready
2022-02-17T05:31:44.008 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com
2022-02-17T05:31:44.056 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045
2022-02-17T05:31:44.169 INFO:teuthology.provision.fog.smithi042:Node is ready
2022-02-17T05:31:44.175 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042.front.sepia.ceph.com
2022-02-17T05:31:44.183 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42 smithi042.front.sepia.ceph.com smithi042
2022-02-17T05:31:44.237 INFO:teuthology.provision.fog.smithi045:Deploy complete!
2022-02-17T05:31:44.363 INFO:teuthology.provision.fog.smithi042:Deploy complete!
2022-02-17T05:31:56.101 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T05:31:56.572 INFO:teuthology.provision.fog.smithi142:Node is ready
2022-02-17T05:31:56.619 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142.front.sepia.ceph.com
2022-02-17T05:31:56.627 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142 smithi142.front.sepia.ceph.com smithi142
2022-02-17T05:31:56.797 INFO:teuthology.provision.fog.smithi142:Deploy complete!
2022-02-17T05:31:57.034 INFO:teuthology.lock.ops:Checking smithi042.front.sepia.ceph.com
2022-02-17T05:31:57.035 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T05:31:57.056 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com
2022-02-17T05:31:57.056 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T05:31:57.073 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com
2022-02-17T05:31:57.073 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T05:31:57.090 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: set os type and version on server
2022-02-17T05:31:57.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com'
2022-02-17T05:31:57.351 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64
2022-02-17T05:31:57.358 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Ubuntu"
2022-02-17T05:31:57.359 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T05:31:57.359 INFO:teuthology.orchestra.run.smithi042.stdout:ID=ubuntu
2022-02-17T05:31:57.359 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE=debian
2022-02-17T05:31:57.359 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T05:31:57.360 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="20.04"
2022-02-17T05:31:57.360 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T05:31:57.360 INFO:teuthology.orchestra.run.smithi042.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T05:31:57.361 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T05:31:57.361 INFO:teuthology.orchestra.run.smithi042.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T05:31:57.361 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_CODENAME=focal
2022-02-17T05:31:57.361 INFO:teuthology.orchestra.run.smithi042.stdout:UBUNTU_CODENAME=focal
2022-02-17T05:31:57.362 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server
2022-02-17T05:31:57.380 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server
2022-02-17T05:31:57.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com'
2022-02-17T05:31:57.691 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64
2022-02-17T05:31:57.697 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Ubuntu"
2022-02-17T05:31:57.698 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T05:31:57.698 INFO:teuthology.orchestra.run.smithi045.stdout:ID=ubuntu
2022-02-17T05:31:57.698 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE=debian
2022-02-17T05:31:57.698 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T05:31:57.698 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="20.04"
2022-02-17T05:31:57.699 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T05:31:57.699 INFO:teuthology.orchestra.run.smithi045.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T05:31:57.699 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T05:31:57.699 INFO:teuthology.orchestra.run.smithi045.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T05:31:57.700 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_CODENAME=focal
2022-02-17T05:31:57.700 INFO:teuthology.orchestra.run.smithi045.stdout:UBUNTU_CODENAME=focal
2022-02-17T05:31:57.700 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server
2022-02-17T05:31:57.718 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server
2022-02-17T05:31:57.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com'
2022-02-17T05:31:57.888 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64
2022-02-17T05:31:57.935 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Ubuntu"
2022-02-17T05:31:57.936 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T05:31:57.936 INFO:teuthology.orchestra.run.smithi142.stdout:ID=ubuntu
2022-02-17T05:31:57.936 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE=debian
2022-02-17T05:31:57.937 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T05:31:57.937 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="20.04"
2022-02-17T05:31:57.937 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T05:31:57.937 INFO:teuthology.orchestra.run.smithi142.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T05:31:57.937 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T05:31:57.938 INFO:teuthology.orchestra.run.smithi142.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T05:31:57.938 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_CODENAME=focal
2022-02-17T05:31:57.938 INFO:teuthology.orchestra.run.smithi142.stdout:UBUNTU_CODENAME=focal
2022-02-17T05:31:57.939 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server
2022-02-17T05:31:58.035 INFO:teuthology.dispatcher.supervisor:Running job 6690423
2022-02-17T05:31:58.036 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-17_04:56:06-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690423 --name benhanokh-2022-02-17_04:56:06-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2022-02-17_04:56:06-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690423/orig.config.yaml
2022-02-17T05:31:58.039 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-17_04:56:06-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690423
2022-02-17T05:31:58.040 INFO:teuthology.dispatcher.supervisor:Job PID: 3608
2022-02-17T05:31:58.040 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-17T12:02:28.387 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-17T12:02:28.637 INFO:teuthology.kill:Killing Pids: {3608}
2022-02-17T12:02:29.842 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-02-17T12:02:29.843 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-02-17T12:02:29.843 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-02-17T12:02:29.844 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:02:29.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com'
2022-02-17T12:02:30.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com'
2022-02-17T12:02:30.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com'
2022-02-17T12:02:30.484 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:02:31.208 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:02:48.762 INFO:teuthology.kill:No teuthology processes running
2022-02-17T12:02:49.988 INFO:teuthology.kill:Nuking machines: ['smithi042', 'smithi045', 'smithi142']
2022-02-17T12:02:50.954 INFO:teuthology.kill:2022-02-17 12:02:50,954.954 INFO:teuthology.nuke:targets:
2022-02-17T12:02:50.956 INFO:teuthology.kill:  smithi042.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/Zk79nKCHbozJOtG6KCNa9aqIwoTktm2lBwnxhjPUUaEg92cXLYsiddSSJiV0vtYMUxs8gtXON/CK3HhlLL0002gXmjMzxOWEvhaBcrq6yYXjfUH8svButzIoHDyu+GqNmwvLVbQ4pHvxFTj5ypBNQfBTyvEXBjzF+g4VXA4XTPcKErqwgW3wKoA8PJSuTbgPhhdBnNFuK8elQdgL+6eTzWGKNNjC15VeNyeiAwVS1eo4ZJl339t1vuRs2qDli11VT5VWqB2alaoenUvCc8jiwoTwkOXaz3Ib+8spWjSl0u3OWR0och6FfyUsWQ/aOxtLgCRV9ui6HLhINVxJnmZ9v9EvvF95vak1y9SDi/Jbsc7zT1mD8KpDmcyZItdjI+pDvqf3O/1aIzEKyVLS0sQaZPzPeSq6tBSQbY3X4oDsP/VG1udj1DOWvSv37iaIQcrt19tHDPM8aqRm0qQCNoeic9SCTmc68EJHvdB5ZUXA3zsBO/uoVAlasnxguwXvQzc=
2022-02-17T12:02:50.956 INFO:teuthology.kill:  smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDekpKTvSjfp6jAujfAR7eYGaHRlYfKO8qyptk1rfPhdM9mQMDGbJrm+uKrDDGw/+fQSrOUtitHY7zKHMz0uj1CDJp6hYRgi+q6rK5HQyK2FL5byayt6AeG6ORnL4yc8LBm6Bh3iBW/QqwDXzG+/PZatwlyjLHwrm1WDdvI5zUHC4/NQ1J1GmZ3hfEvpE7RSal6g+3Fmjw02ks9IbmKy4Jffj3UkNkkdv/EafVJeENcwOK/hSpL2bEhkifIeAXGE+OowLCjVtfJTkm3loMpfUT8VfrB7ROfv8kJ7830BiwGPL/eS5C2WD0I27DnxPOziqMEwq0tRmslLVYVhsIGX0LkJneMpUU1ZBgIgAv9FJrgYpczCs4B5MOMjih7I2/8gTdYnuQ7ttyi5pVlZXiO6iXPUvquTaVBHYf1tiJCHv3hz7v902KIYc4vGuAiO3JlU5DLc8oYdUrx64xCKprECTZJOjq11UY9YTkRXxpEhXOO1kq326dT8PaC95QCvTVGLPE=
2022-02-17T12:02:50.956 INFO:teuthology.kill:  smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvJAfC0hQVhfQwFVDC0oWWTJpfKVFzyapnX3bunTmczOi3Sdokdm0IjH+N6YNMjUuVVeCxEeQye7+lsp3lCWv/onGSJijaYkr4RVn4bjHNxPnBLrsiVtygInEvAhSV/H116F7WKfecSbO5pFo1RapYHPvNbp6iaa2ahGp5v88aXT8ApsYB7HExQNYvE6jHkDY+EPN9OP3DOA1+32uGY1538cWu2qAqpSxdoM/XSIQ5lqHA/Sy4qs/oVb/Lwl4n4xlNjLryl9hFO7pQmUIGtD8X0IysKa1U5R1j9ltp3ow1r6ABZLjXmlL1OtX0I1ZXXxjpnF9fNgOhAqq3d3LlK7N6783jOT72N8c3eCeOnz8akMFUQbtb81ZoFbOef2G/dNqPLk7LHFLo+u6nQW/WzZkcHY3yCqMIPMkbYklu9AnP1/gJ4VisSSdXFswyt6zZBxP3cYlqWd6Xftdxji3erCtStj6YalooFagEMEWhHu/S0gQvOuHBv69yoQGY1vu0d/s=
2022-02-17T12:02:50.982 INFO:teuthology.kill:2022-02-17 12:02:50,982.982 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:02:50.986 INFO:teuthology.kill:2022-02-17 12:02:50,986.986 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:02:50.988 INFO:teuthology.kill:2022-02-17 12:02:50,988.988 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:02:51.069 INFO:teuthology.kill:2022-02-17 12:02:51,069.069 INFO:teuthology.orchestra.console:Power off smithi045
2022-02-17T12:02:51.083 INFO:teuthology.kill:2022-02-17 12:02:51,082.082 INFO:teuthology.orchestra.console:Power off smithi042
2022-02-17T12:02:51.094 INFO:teuthology.kill:2022-02-17 12:02:51,094.094 INFO:teuthology.orchestra.console:Power off smithi142
2022-02-17T12:02:59.454 INFO:teuthology.kill:2022-02-17 12:02:59,453.453 INFO:teuthology.orchestra.console:Power off for smithi045 completed
2022-02-17T12:02:59.458 INFO:teuthology.kill:2022-02-17 12:02:59,458.458 INFO:teuthology.orchestra.console:Power off for smithi042 completed
2022-02-17T12:02:59.605 INFO:teuthology.kill:2022-02-17 12:02:59,605.605 INFO:teuthology.lock.ops:unlocked: smithi042.front.sepia.ceph.com
2022-02-17T12:02:59.609 INFO:teuthology.kill:2022-02-17 12:02:59,609.609 INFO:teuthology.lock.ops:unlocked: smithi045.front.sepia.ceph.com
2022-02-17T12:03:03.575 INFO:teuthology.kill:2022-02-17 12:03:03,575.575 INFO:teuthology.orchestra.console:Power off for smithi142 completed
2022-02-17T12:03:03.719 INFO:teuthology.kill:2022-02-17 12:03:03,719.719 INFO:teuthology.lock.ops:unlocked: smithi142.front.sepia.ceph.com
2022-02-17T12:05:04.027 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-17T12:05:04.146 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi045 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-17_11:32:42-rados-safe_shutdown_v2_compact_log_async-distro-basic-smithi/6690975
2022-02-17T12:05:04.147 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi142 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-17_11:32:42-rados-safe_shutdown_v2_compact_log_async-distro-basic-smithi/6690975