2022-02-17T06:33:27.847 INFO:root:teuthology version: 1.1.0-c6442aa9
2022-02-17T06:33:28.006 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi099.front.sepia.ceph.com' reimaging
2022-02-17T06:33:28.007 INFO:teuthology.lock.ops:Updating [ubuntu@smithi099.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T06:33:28.007 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2022-02-17T06:33:28.027 INFO:teuthology.lock.ops:Node 'ubuntu@smithi099.front.sepia.ceph.com' reimaging is complete
2022-02-17T06:33:28.028 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging
2022-02-17T06:33:28.028 INFO:teuthology.lock.ops:Updating [ubuntu@smithi136.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T06:33:28.028 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server
2022-02-17T06:33:28.051 INFO:teuthology.lock.ops:Node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging is complete
2022-02-17T06:33:28.052 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi197.front.sepia.ceph.com' reimaging
2022-02-17T06:33:28.052 INFO:teuthology.lock.ops:Updating [ubuntu@smithi197.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T06:33:28.052 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server
2022-02-17T06:33:28.072 INFO:teuthology.lock.ops:Node 'ubuntu@smithi197.front.sepia.ceph.com' reimaging is complete
2022-02-17T06:33:28.255 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of ubuntu 20.04
2022-02-17T06:33:28.272 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of ubuntu 20.04
2022-02-17T06:33:28.286 INFO:teuthology.provision.fog.smithi197:Scheduling deploy of ubuntu 20.04
2022-02-17T06:33:28.571 INFO:teuthology.orchestra.console:Power off smithi099
2022-02-17T06:33:28.588 INFO:teuthology.orchestra.console:Power off smithi136
2022-02-17T06:33:28.603 INFO:teuthology.orchestra.console:Power off smithi197
2022-02-17T06:33:28.735 INFO:teuthology.orchestra.console:Power off for smithi136 completed
2022-02-17T06:33:28.736 INFO:teuthology.orchestra.console:Power off for smithi099 completed
2022-02-17T06:33:28.745 INFO:teuthology.orchestra.console:Power off for smithi197 completed
2022-02-17T06:33:28.837 INFO:teuthology.orchestra.console:Power on smithi136
2022-02-17T06:33:28.847 INFO:teuthology.orchestra.console:Power on smithi099
2022-02-17T06:33:28.855 INFO:teuthology.orchestra.console:Power on smithi197
2022-02-17T06:33:37.214 INFO:teuthology.orchestra.console:Power on for smithi136 completed
2022-02-17T06:33:37.228 INFO:teuthology.orchestra.console:Power on for smithi197 completed
2022-02-17T06:33:37.316 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish
2022-02-17T06:33:37.320 INFO:teuthology.orchestra.console:Power on for smithi099 completed
2022-02-17T06:33:37.329 INFO:teuthology.provision.fog.smithi197:Waiting for deploy to finish
2022-02-17T06:33:37.421 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish
2022-02-17T06:38:26.808 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T06:38:27.296 INFO:teuthology.provision.fog.smithi197:Node is ready
2022-02-17T06:38:27.345 INFO:teuthology.orchestra.run.smithi197.stdout:smithi197.front.sepia.ceph.com
2022-02-17T06:38:27.352 INFO:teuthology.orchestra.run.smithi197.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197
2022-02-17T06:38:27.548 INFO:teuthology.provision.fog.smithi197:Deploy complete!
2022-02-17T06:38:37.468 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T06:38:37.963 INFO:teuthology.provision.fog.smithi099:Node is ready
2022-02-17T06:38:37.969 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com
2022-02-17T06:38:37.976 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099
2022-02-17T06:38:38.182 INFO:teuthology.provision.fog.smithi099:Deploy complete!
2022-02-17T06:38:43.690 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T06:38:44.184 INFO:teuthology.provision.fog.smithi136:Node is ready
2022-02-17T06:38:44.192 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com
2022-02-17T06:38:44.243 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136
2022-02-17T06:38:44.478 INFO:teuthology.provision.fog.smithi136:Deploy complete!
2022-02-17T06:38:44.715 INFO:teuthology.lock.ops:Checking smithi099.front.sepia.ceph.com
2022-02-17T06:38:44.716 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T06:38:44.735 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com
2022-02-17T06:38:44.736 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T06:38:44.753 INFO:teuthology.lock.ops:Checking smithi197.front.sepia.ceph.com
2022-02-17T06:38:44.754 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T06:38:44.772 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: set os type and version on server
2022-02-17T06:38:44.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com'
2022-02-17T06:38:44.950 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64
2022-02-17T06:38:44.957 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Ubuntu"
2022-02-17T06:38:44.959 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T06:38:44.959 INFO:teuthology.orchestra.run.smithi099.stdout:ID=ubuntu
2022-02-17T06:38:44.959 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE=debian
2022-02-17T06:38:44.959 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T06:38:44.960 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="20.04"
2022-02-17T06:38:44.960 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T06:38:44.960 INFO:teuthology.orchestra.run.smithi099.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T06:38:44.960 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T06:38:44.961 INFO:teuthology.orchestra.run.smithi099.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T06:38:44.961 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_CODENAME=focal
2022-02-17T06:38:44.961 INFO:teuthology.orchestra.run.smithi099.stdout:UBUNTU_CODENAME=focal
2022-02-17T06:38:44.962 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2022-02-17T06:38:44.981 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server
2022-02-17T06:38:44.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com'
2022-02-17T06:38:45.154 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64
2022-02-17T06:38:45.202 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu"
2022-02-17T06:38:45.202 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T06:38:45.203 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu
2022-02-17T06:38:45.203 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian
2022-02-17T06:38:45.203 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T06:38:45.203 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="20.04"
2022-02-17T06:38:45.204 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T06:38:45.204 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T06:38:45.204 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T06:38:45.204 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T06:38:45.205 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=focal
2022-02-17T06:38:45.205 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=focal
2022-02-17T06:38:45.205 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server
2022-02-17T06:38:45.223 INFO:teuthology.lock.ops:Updating [smithi197.front.sepia.ceph.com]: set os type and version on server
2022-02-17T06:38:45.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi197.front.sepia.ceph.com'
2022-02-17T06:38:45.441 INFO:teuthology.orchestra.run.smithi197.stdout:x86_64
2022-02-17T06:38:45.448 INFO:teuthology.orchestra.run.smithi197.stdout:NAME="Ubuntu"
2022-02-17T06:38:45.448 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-17T06:38:45.448 INFO:teuthology.orchestra.run.smithi197.stdout:ID=ubuntu
2022-02-17T06:38:45.449 INFO:teuthology.orchestra.run.smithi197.stdout:ID_LIKE=debian
2022-02-17T06:38:45.449 INFO:teuthology.orchestra.run.smithi197.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-17T06:38:45.449 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_ID="20.04"
2022-02-17T06:38:45.449 INFO:teuthology.orchestra.run.smithi197.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-17T06:38:45.449 INFO:teuthology.orchestra.run.smithi197.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-17T06:38:45.450 INFO:teuthology.orchestra.run.smithi197.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-17T06:38:45.450 INFO:teuthology.orchestra.run.smithi197.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-17T06:38:45.450 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_CODENAME=focal
2022-02-17T06:38:45.450 INFO:teuthology.orchestra.run.smithi197.stdout:UBUNTU_CODENAME=focal
2022-02-17T06:38:45.451 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server
2022-02-17T06:38:45.546 INFO:teuthology.dispatcher.supervisor:Running job 6690437
2022-02-17T06:38:45.547 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690437 --name benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690437/orig.config.yaml
2022-02-17T06:38:45.551 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-17_04:58:07-rados-GBH_TEST_scrub_allocation_file_on_mount-distro-basic-smithi/6690437
2022-02-17T06:38:45.552 INFO:teuthology.dispatcher.supervisor:Job PID: 13418
2022-02-17T06:38:45.552 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-17T13:09:15.287 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-17T13:09:15.499 INFO:teuthology.kill:Killing Pids: {13418}
2022-02-17T13:09:16.787 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-02-17T13:09:16.788 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-02-17T13:09:16.788 INFO:teuthology.task.internal:roles: ubuntu@smithi197.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-02-17T13:09:16.788 INFO:teuthology.misc:Compressing logs...
2022-02-17T13:09:16.789 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com'
2022-02-17T13:09:16.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com'
2022-02-17T13:09:17.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi197.front.sepia.ceph.com'
2022-02-17T13:09:17.416 INFO:teuthology.misc:Compressing logs...
2022-02-17T13:09:18.153 INFO:teuthology.misc:Compressing logs...
2022-02-17T13:10:24.489 INFO:teuthology.kill:No teuthology processes running
2022-02-17T13:10:25.702 INFO:teuthology.kill:Nuking machines: ['smithi099', 'smithi136', 'smithi197']
2022-02-17T13:10:26.756 INFO:teuthology.kill:2022-02-17 13:10:26,755.755 INFO:teuthology.nuke:targets:
2022-02-17T13:10:26.757 INFO:teuthology.kill:  smithi099.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTtZrh5H3GsATnrDTPJHFjux1mLa+bxiLnXQakPheNysf1njMsZ/qfthxUUgwwOjJzVmbBJB6LslzvtN4EJ7q9ZwY2WHMrf7a4hZ7ut/jrch+lj8j5xeAWqHKWLX1R37Pq+IcFRwjkDpfQIrUl9BuWapBcQhAMhW2xZGNcbrCLxG07sIs85Qqz5IJwpp+jeOr01GSUsVrEzZr5rZZ4pBjsCsePfApRcLvsZeM7rb9XQnXsbwMGtgUO1cdklFsuMI/7WrIEU3e0s4KPl88Dp8t3nHwFEMmgZvp783gfsOOn0++4yFwI78ocIzbWNFh5pyJ5X2df957+usZ1xGb6/AaocTdNrItVjSijPxZ/PXMconhLo1X5Ly+NcN81qVdXmSeGyw/KitGy7EFY48+e6f3ymRjrRuIalq49DW+yctVNgnASAK4bFrkbUfLcpLFHiko6QshTI5rTg8eu8WphhlVo0mWWeNmxFbieG+jPwxFYy0Nyy4lOyEsKjBM5aEdgUfk=
2022-02-17T13:10:26.757 INFO:teuthology.kill:  smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIUppicl97IeJoHtlttLB4luPQFhttMa9n3K5YkoZwlb0lWNz/ySKr6zvleEFFW/fhuMdBMVfgCuUutfjMnyEUYr2tRK41hWuNr4Vfeotdp0dplXx9BKx3hHUsDfeAle07ZWeMoLCJ5SmrMb4uYwakCPdOthFfg60cJ5lUUYurgDQk/gbjpBW2v/NYOOi76kJdTRltaJhJOWmba1P3rjc3kMQ01+ueZqWvF4x7MMPJwxYBZsWSK0Fz3iQRyhUcjdR9QVzT53XLzgeQ9W4KlvQvH9lK0YwdXQxttbyx2/5n+0muDlKKVwLTTSoBs+8g/2WQL4mio0Gc68PR3etgkWH1NseN1Ig5Jp40qeteUtbOfbSKWOzZnDyvQDbKplcFgprzr7LndeZPwVOiqw1bIQTWJL5F9PKOCtRvylqguffZpYSPd2aaAhDbG8P3lUy02HExqv+xibdxoUvf6Gm3mS1FvK6EpyjMsuFOTXUJG6fynmFIsNbdJe3OECxEbhRGF2s=
2022-02-17T13:10:26.757 INFO:teuthology.kill:  smithi197.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR/vpW9Lcm0g7ffuwulNhqdaSwZQX3SzmlrG05RxSbcgW+CyXzGKkbjAis0mQL+xmtoiQUeMsF+61Q9ysLhMVCeITS5IGn+8P9q7X9awrOKkjlttM2OvXAfC2URKeZp+JusTE+V0O1+7HeAExa1VwmFhYtfn7tb/RlAzjYJQJWAFk5goXMlr09NY4xRIM4XdOYKWTi4m+5JxJ00sn/Q2d2K5o+zSQZcrbk5doJk2mjJCpntE9p/X2F8VJjnp3N41uxpKPxlQpFQeJTI60JJyVFJlSDRwzm25STKKM+9M87N8QwMWLxYfK6W1zcyOoMjhZIouI8z5bDhwkPcUM/jEFtcpXkGVQcBev0OSRb3PKsSQG2TjRLranJN++tQBaHRY4VRCh6uw0aMWiuoExyTXpHmvrS1NZg4it1EBrZrDJ6RdDqSMi0aCz2mlxXERmDIjjTS10Fib4XY2Q1+STVsYuwju/pkAg1EcGnFkqXS55UGm0KPuikNIQ8E24/+XKHboM=
2022-02-17T13:10:26.788 INFO:teuthology.kill:2022-02-17 13:10:26,788.788 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T13:10:26.796 INFO:teuthology.kill:2022-02-17 13:10:26,795.795 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T13:10:26.799 INFO:teuthology.kill:2022-02-17 13:10:26,799.799 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T13:10:26.889 INFO:teuthology.kill:2022-02-17 13:10:26,888.888 INFO:teuthology.orchestra.console:Power off smithi197
2022-02-17T13:10:26.905 INFO:teuthology.kill:2022-02-17 13:10:26,904.904 INFO:teuthology.orchestra.console:Power off smithi136
2022-02-17T13:10:26.916 INFO:teuthology.kill:2022-02-17 13:10:26,915.915 INFO:teuthology.orchestra.console:Power off smithi099
2022-02-17T13:10:39.384 INFO:teuthology.kill:2022-02-17 13:10:39,384.384 INFO:teuthology.orchestra.console:Power off for smithi197 completed
2022-02-17T13:10:39.390 INFO:teuthology.kill:2022-02-17 13:10:39,389.389 INFO:teuthology.orchestra.console:Power off for smithi136 completed
2022-02-17T13:10:39.397 INFO:teuthology.kill:2022-02-17 13:10:39,397.397 INFO:teuthology.orchestra.console:Power off for smithi099 completed
2022-02-17T13:10:39.537 INFO:teuthology.kill:2022-02-17 13:10:39,536.536 INFO:teuthology.lock.ops:unlocked: smithi136.front.sepia.ceph.com
2022-02-17T13:10:39.539 INFO:teuthology.kill:2022-02-17 13:10:39,539.539 INFO:teuthology.lock.ops:unlocked: smithi197.front.sepia.ceph.com
2022-02-17T13:10:39.543 INFO:teuthology.kill:2022-02-17 13:10:39,543.543 INFO:teuthology.lock.ops:unlocked: smithi099.front.sepia.ceph.com
2022-02-17T13:12:39.875 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-17T13:12:40.002 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi099 but it was locked by another job: /home/teuthworker/archive/rfriedma-2022-02-17_12:48:58-rados:thrash-wip-rf-scrubBEv1_2-distro-basic-smithi/6690997
2022-02-17T13:12:40.002 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi197 but it was locked by another job: /home/teuthworker/archive/rfriedma-2022-02-17_12:48:58-rados:thrash-wip-rf-scrubBEv1_2-distro-basic-smithi/6690997