2022-02-17T06:21:44.857 INFO:root:teuthology version: 1.1.0-c6442aa9
2022-02-17T06:21:45.017 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi137.front.sepia.ceph.com' reimaging
2022-02-17T06:21:45.019 INFO:teuthology.lock.ops:Updating [ubuntu@smithi137.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T06:21:45.019 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server
2022-02-17T06:21:45.038 INFO:teuthology.lock.ops:Node 'ubuntu@smithi137.front.sepia.ceph.com' reimaging is complete
2022-02-17T06:21:45.039 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging
2022-02-17T06:21:45.039 INFO:teuthology.lock.ops:Updating [ubuntu@smithi176.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T06:21:45.039 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server
2022-02-17T06:21:45.059 INFO:teuthology.lock.ops:Node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging is complete
2022-02-17T06:21:45.059 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi181.front.sepia.ceph.com' reimaging
2022-02-17T06:21:45.060 INFO:teuthology.lock.ops:Updating [ubuntu@smithi181.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T06:21:45.060 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2022-02-17T06:21:45.078 INFO:teuthology.lock.ops:Node 'ubuntu@smithi181.front.sepia.ceph.com' reimaging is complete
2022-02-17T06:21:45.252 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of ubuntu 20.04
2022-02-17T06:21:45.295 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of ubuntu 20.04
2022-02-17T06:21:45.305 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of ubuntu 20.04
2022-02-17T06:21:45.577 INFO:teuthology.orchestra.console:Power off smithi137
2022-02-17T06:21:45.619 INFO:teuthology.orchestra.console:Power off smithi176
2022-02-17T06:21:45.632 INFO:teuthology.orchestra.console:Power off smithi181
2022-02-17T06:21:45.742 INFO:teuthology.orchestra.console:Power off for smithi137 completed
2022-02-17T06:21:45.764 INFO:teuthology.orchestra.console:Power off for smithi176 completed
2022-02-17T06:21:45.776 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2022-02-17T06:21:45.843 INFO:teuthology.orchestra.console:Power on smithi137
2022-02-17T06:21:45.871 INFO:teuthology.orchestra.console:Power on smithi176
2022-02-17T06:21:45.877 INFO:teuthology.orchestra.console:Power on smithi181
2022-02-17T06:21:50.101 INFO:teuthology.orchestra.console:Power on for smithi137 completed
2022-02-17T06:21:50.128 INFO:teuthology.orchestra.console:Power on for smithi181 completed
2022-02-17T06:21:50.203 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish
2022-02-17T06:21:50.229 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish
2022-02-17T06:21:54.339 INFO:teuthology.orchestra.console:Power on for smithi176 completed
2022-02-17T06:21:54.441 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish
2022-02-17T06:26:47.366 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T06:26:47.874 INFO:teuthology.provision.fog.smithi181:Node is ready
2022-02-17T06:26:47.880 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com
2022-02-17T06:26:47.930 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181
2022-02-17T06:26:48.107 INFO:teuthology.provision.fog.smithi181:Deploy complete!
2022-02-17T06:26:56.316 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T06:26:56.805 INFO:teuthology.provision.fog.smithi137:Node is ready
2022-02-17T06:26:56.812 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com
2022-02-17T06:26:56.820 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137
2022-02-17T06:26:56.940 INFO:teuthology.provision.fog.smithi137:Deploy complete!
2022-02-17T06:27:00.683 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T06:27:01.152 INFO:teuthology.provision.fog.smithi176:Node is ready
2022-02-17T06:27:01.158 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com
2022-02-17T06:27:01.208 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176
2022-02-17T06:27:01.372 INFO:teuthology.provision.fog.smithi176:Deploy complete!
2022-02-17T06:27:01.624 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com
2022-02-17T06:27:01.625 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T06:27:01.644 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com
2022-02-17T06:27:01.644 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T06:27:01.661 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com
2022-02-17T06:27:01.661 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T06:27:01.680 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server
2022-02-17T06:27:01.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com'
2022-02-17T06:27:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64
2022-02-17T06:27:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Ubuntu"
2022-02-17T06:27:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T06:27:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:ID=ubuntu
2022-02-17T06:27:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE=debian
2022-02-17T06:27:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T06:27:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="20.04"
2022-02-17T06:27:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T06:27:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T06:27:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T06:27:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T06:27:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_CODENAME=focal
2022-02-17T06:27:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:UBUNTU_CODENAME=focal
2022-02-17T06:27:01.866 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server
2022-02-17T06:27:01.885 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server
2022-02-17T06:27:01.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com'
2022-02-17T06:27:02.051 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64
2022-02-17T06:27:02.057 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Ubuntu"
2022-02-17T06:27:02.058 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T06:27:02.058 INFO:teuthology.orchestra.run.smithi176.stdout:ID=ubuntu
2022-02-17T06:27:02.058 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE=debian
2022-02-17T06:27:02.059 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T06:27:02.059 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="20.04"
2022-02-17T06:27:02.059 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T06:27:02.059 INFO:teuthology.orchestra.run.smithi176.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T06:27:02.060 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T06:27:02.060 INFO:teuthology.orchestra.run.smithi176.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T06:27:02.060 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_CODENAME=focal
2022-02-17T06:27:02.060 INFO:teuthology.orchestra.run.smithi176.stdout:UBUNTU_CODENAME=focal
2022-02-17T06:27:02.061 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server
2022-02-17T06:27:02.078 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server
2022-02-17T06:27:02.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2022-02-17T06:27:02.342 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64
2022-02-17T06:27:02.349 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Ubuntu"
2022-02-17T06:27:02.349 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T06:27:02.350 INFO:teuthology.orchestra.run.smithi181.stdout:ID=ubuntu
2022-02-17T06:27:02.350 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE=debian
2022-02-17T06:27:02.350 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T06:27:02.351 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="20.04"
2022-02-17T06:27:02.351 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T06:27:02.351 INFO:teuthology.orchestra.run.smithi181.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T06:27:02.351 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T06:27:02.352 INFO:teuthology.orchestra.run.smithi181.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T06:27:02.352 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_CODENAME=focal
2022-02-17T06:27:02.352 INFO:teuthology.orchestra.run.smithi181.stdout:UBUNTU_CODENAME=focal
2022-02-17T06:27:02.353 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2022-02-17T06:27:02.447 INFO:teuthology.dispatcher.supervisor:Running job 6690433
2022-02-17T06:27:02.447 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690433 --name benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690433/orig.config.yaml
2022-02-17T06:27:02.451 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690433
2022-02-17T06:27:02.451 INFO:teuthology.dispatcher.supervisor:Job PID: 15206
2022-02-17T06:27:02.452 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-17T12:57:31.815 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-17T12:57:32.015 INFO:teuthology.kill:Killing Pids: {15206}
2022-02-17T12:57:33.193 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-02-17T12:57:33.194 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-02-17T12:57:33.195 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-02-17T12:57:33.195 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:57:33.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com'
2022-02-17T12:57:33.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com'
2022-02-17T12:57:33.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2022-02-17T12:57:33.843 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:57:34.593 INFO:teuthology.misc:Compressing logs...
2022-02-17T12:57:53.158 INFO:teuthology.kill:No teuthology processes running
2022-02-17T12:57:54.356 INFO:teuthology.kill:Nuking machines: ['smithi137', 'smithi176', 'smithi181']
2022-02-17T12:57:55.405 INFO:teuthology.kill:2022-02-17 12:57:55,405.405 INFO:teuthology.nuke:targets:
2022-02-17T12:57:55.406 INFO:teuthology.kill:  smithi137.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZalGbg9pDA1EsRAbG1ppwWEUBnMx+GWqGLvyFpYDR29ps/MqZaIQ8rfs24K+vioeM8If8j/G87X6KyxVi6zHAK+mr6rFjrw0klYUsH+dP/dn1whhwjU7aYhK0wMBlTwd2OcmNEfkHod/SaEhJEVOFteoF30UFdVuokLGVrh1ERkRiUg6OF13UvpNmjLI+KA3L2xjxlhF3dlkrEue3Ix0wtz2sRkuzpaNLxw8iv0v5IZy815qYdcmVEIrkmDzBkQCtwIcEr5f2Y6faG3fegZhln15VG98bQ2sfbMZmZov0PSqByS8/c6xs8TZaUPNF6MrnQeJcobCwQD0xCEvUXgdid/f299xzZmp28wNGhCV1GxhTGRWuAo8FEsf9iJ9TJ0lhLrjKQ/3jul6P3mhgmu4edK9fOH/Ysa+IVykvLoCprAOlbPS5O3hGfCQTX0+je04iyRlyCqS/GOWy9BB9E0tZSPQbe5eRlUQiRoXDkFFVuaILY/UQSXoho1t84ovrgj0=
2022-02-17T12:57:55.407 INFO:teuthology.kill:  smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWie65SsU0CUjfglPmNjQqhHtux+6D6HoHc8+WYAXLBLODLL1pK/6Dz+KvU9x6Lg8fitA9hJLM2hfduueV8vgAok1910TXy8UKFJvD1PGUU7Clrrllm/mBL0YfxB8SRE6PilWZo2oY8zV5hjHFWzvCGM53y/SMJpY1wmuGJGScNHdkxnaFhybTyXYsn1fMAIvZEAms89ZGaF510xwm15FlI8rAB+Drq38MFvnS4ISZdsQSoro5gu6iWFEzsfOTCF92FXTDMGKojiaEjjSmbzX8bnvMFskkd+53/xb+q2fhkM7Kv6oAUK2sl2gf8sAcgMLzln2r/3pk2LqBPEI7Rjz4pfz7qsW9zFt2dSH8Vo8xSX/4eU6VTfiilMczOIHgtYDB5UX8vhaFP3ayLxtdHqj+JsLHw9oj3qL5Tjbm6/KSYUAc5vJMlSnKpoZG4pNzG2BmxtYdy/aWDXE32yxbmdZCboB0aQsLxtSAi1jhPYj9S8EoicRzs5Asey5r3MCvXG8=
2022-02-17T12:57:55.407 INFO:teuthology.kill:  smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgPhkJvdXPs4aSlqcEkNBQnKiQG6h76551vDVfHZCng6EJHWw8xWA5lZiXhBnt8K60RbqwTfQza0V4vrbfnvKtw9mBGiDdpe5y0jc8bfzddKhmIyL3vwQeEVA1bhOQjIo9npKyUn5/Uebe2NhRRnQoXaCr2QG90mg2kqU+IDsludZ3QJn8hCcjbHgHWJh4phtXZfSl8F80TXlpTn5mXcDKzSmgJ/+/Jm6Ryx0f6ud/BmcTpZ4GMA2DFn7UDulT4S+Z7jvLzNVjORjmiwit2B4BrxaJKwupydYEx5oWSkMawTahFW/rURIp1Ji/AbQR+4FovZiE81lmBxouV2J6nJpDQj2UMYEF7VbviSS1wIgeQ+7Z3YxusuzK2zN5VWIGX68kcXiVjnDw4fFKJT82rXmY48/cRygbjL1fg7uGcEV8BsnkJgMjhpWd1oK+AqNMk69MQX/6dOtuwuO4zEEPUOPeieP1/hIk/QHhWmJmc1zGLOqhw9NtYa3kZaR5X5bTBdU=
2022-02-17T12:57:55.433 INFO:teuthology.kill:2022-02-17 12:57:55,433.433 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:57:55.437 INFO:teuthology.kill:2022-02-17 12:57:55,437.437 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:57:55.442 INFO:teuthology.kill:2022-02-17 12:57:55,442.442 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T12:57:55.532 INFO:teuthology.kill:2022-02-17 12:57:55,531.531 INFO:teuthology.orchestra.console:Power off smithi137
2022-02-17T12:57:55.550 INFO:teuthology.kill:2022-02-17 12:57:55,550.550 INFO:teuthology.orchestra.console:Power off smithi181
2022-02-17T12:57:55.589 INFO:teuthology.kill:2022-02-17 12:57:55,589.589 INFO:teuthology.orchestra.console:Power off smithi176
2022-02-17T12:58:03.917 INFO:teuthology.kill:2022-02-17 12:58:03,916.916 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2022-02-17T12:58:03.949 INFO:teuthology.kill:2022-02-17 12:58:03,949.949 INFO:teuthology.orchestra.console:Power off for smithi176 completed
2022-02-17T12:58:04.066 INFO:teuthology.kill:2022-02-17 12:58:04,065.065 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com
2022-02-17T12:58:04.095 INFO:teuthology.kill:2022-02-17 12:58:04,095.095 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com
2022-02-17T12:58:08.016 INFO:teuthology.kill:2022-02-17 12:58:08,016.016 INFO:teuthology.orchestra.console:Power off for smithi137 completed
2022-02-17T12:58:08.161 INFO:teuthology.kill:2022-02-17 12:58:08,161.161 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com
2022-02-17T13:00:08.470 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-17T13:00:08.595 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi137 but it was locked by another job: /home/teuthworker/archive/nojha-2022-02-16_23:40:43-rados:thrash-GBH_scrub_allocation_file_on_mount-distro-basic-smithi/6690240
2022-02-17T13:00:08.596 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi176 but it was locked by another job: /home/teuthworker/archive/rfriedma-2022-02-17_12:48:58-rados:thrash-wip-rf-scrubBEv1_2-distro-basic-smithi/6690991
2022-02-17T13:00:08.596 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi181 but it was locked by another job: /home/teuthworker/archive/rfriedma-2022-02-17_12:48:58-rados:thrash-wip-rf-scrubBEv1_2-distro-basic-smithi/6690991