2022-01-13T07:13:31.230 INFO:root:teuthology version: 1.1.0-6fc20113
2022-01-13T07:13:31.388 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging
2022-01-13T07:13:31.389 INFO:teuthology.lock.ops:Updating [ubuntu@smithi165.front.sepia.ceph.com]: reset os type and version on server
2022-01-13T07:13:31.389 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server
2022-01-13T07:13:31.411 INFO:teuthology.lock.ops:Node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging is complete
2022-01-13T07:13:31.412 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi189.front.sepia.ceph.com' reimaging
2022-01-13T07:13:31.412 INFO:teuthology.lock.ops:Updating [ubuntu@smithi189.front.sepia.ceph.com]: reset os type and version on server
2022-01-13T07:13:31.412 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server
2022-01-13T07:13:31.434 INFO:teuthology.lock.ops:Node 'ubuntu@smithi189.front.sepia.ceph.com' reimaging is complete
2022-01-13T07:13:31.435 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi192.front.sepia.ceph.com' reimaging
2022-01-13T07:13:31.435 INFO:teuthology.lock.ops:Updating [ubuntu@smithi192.front.sepia.ceph.com]: reset os type and version on server
2022-01-13T07:13:31.435 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server
2022-01-13T07:13:31.459 INFO:teuthology.lock.ops:Node 'ubuntu@smithi192.front.sepia.ceph.com' reimaging is complete
2022-01-13T07:13:31.623 INFO:teuthology.provision.fog.smithi165:Scheduling deploy of ubuntu 20.04
2022-01-13T07:13:31.650 INFO:teuthology.provision.fog.smithi189:Scheduling deploy of ubuntu 20.04
2022-01-13T07:13:31.664 INFO:teuthology.provision.fog.smithi192:Scheduling deploy of ubuntu 20.04
2022-01-13T07:13:31.950 INFO:teuthology.orchestra.console:Power off smithi165
2022-01-13T07:13:31.990 INFO:teuthology.orchestra.console:Power off smithi189
2022-01-13T07:13:32.026 INFO:teuthology.orchestra.console:Power off smithi192
2022-01-13T07:13:40.328 INFO:teuthology.orchestra.console:Power off for smithi165 completed
2022-01-13T07:13:40.430 INFO:teuthology.orchestra.console:Power on smithi165
2022-01-13T07:13:44.469 INFO:teuthology.orchestra.console:Power off for smithi189 completed
2022-01-13T07:13:44.499 INFO:teuthology.orchestra.console:Power off for smithi192 completed
2022-01-13T07:13:44.571 INFO:teuthology.orchestra.console:Power on smithi189
2022-01-13T07:13:44.601 INFO:teuthology.orchestra.console:Power on smithi192
2022-01-13T07:13:48.828 INFO:teuthology.orchestra.console:Power on for smithi189 completed
2022-01-13T07:13:48.930 INFO:teuthology.provision.fog.smithi189:Waiting for deploy to finish
2022-01-13T07:13:48.941 INFO:teuthology.orchestra.console:Power on for smithi165 completed
2022-01-13T07:13:49.043 INFO:teuthology.provision.fog.smithi165:Waiting for deploy to finish
2022-01-13T07:13:52.971 INFO:teuthology.orchestra.console:Power on for smithi192 completed
2022-01-13T07:13:53.072 INFO:teuthology.provision.fog.smithi192:Waiting for deploy to finish
2022-01-13T07:19:12.770 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-13T07:19:13.301 INFO:teuthology.provision.fog.smithi192:Node is ready
2022-01-13T07:19:13.308 INFO:teuthology.orchestra.run.smithi192.stdout:smithi192.front.sepia.ceph.com
2022-01-13T07:19:13.317 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.15.192 smithi192.front.sepia.ceph.com smithi192
2022-01-13T07:19:13.463 INFO:teuthology.provision.fog.smithi192:Deploy complete!
2022-01-13T07:19:20.733 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-13T07:19:21.295 INFO:teuthology.provision.fog.smithi189:Node is ready
2022-01-13T07:19:21.301 INFO:teuthology.orchestra.run.smithi189.stdout:smithi189.front.sepia.ceph.com
2022-01-13T07:19:21.351 INFO:teuthology.orchestra.run.smithi189.stdout:172.21.15.189 smithi189.front.sepia.ceph.com smithi189
2022-01-13T07:19:21.582 INFO:teuthology.provision.fog.smithi189:Deploy complete!
2022-01-13T07:19:23.822 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-13T07:19:24.374 INFO:teuthology.provision.fog.smithi165:Node is ready
2022-01-13T07:19:24.381 INFO:teuthology.orchestra.run.smithi165.stdout:smithi165.front.sepia.ceph.com
2022-01-13T07:19:24.388 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165 smithi165.front.sepia.ceph.com smithi165
2022-01-13T07:19:24.604 INFO:teuthology.provision.fog.smithi165:Deploy complete!
2022-01-13T07:19:24.843 INFO:teuthology.lock.ops:Checking smithi165.front.sepia.ceph.com
2022-01-13T07:19:24.844 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-13T07:19:24.862 INFO:teuthology.lock.ops:Checking smithi189.front.sepia.ceph.com
2022-01-13T07:19:24.863 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-13T07:19:24.881 INFO:teuthology.lock.ops:Checking smithi192.front.sepia.ceph.com
2022-01-13T07:19:24.882 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-13T07:19:24.904 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: set os type and version on server
2022-01-13T07:19:24.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com'
2022-01-13T07:19:25.082 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64
2022-01-13T07:19:25.133 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="Ubuntu"
2022-01-13T07:19:25.134 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-01-13T07:19:25.134 INFO:teuthology.orchestra.run.smithi165.stdout:ID=ubuntu
2022-01-13T07:19:25.134 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE=debian
2022-01-13T07:19:25.135 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-01-13T07:19:25.135 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="20.04"
2022-01-13T07:19:25.135 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://www.ubuntu.com/"
2022-01-13T07:19:25.135 INFO:teuthology.orchestra.run.smithi165.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-01-13T07:19:25.135 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-01-13T07:19:25.136 INFO:teuthology.orchestra.run.smithi165.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-01-13T07:19:25.136 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_CODENAME=focal
2022-01-13T07:19:25.136 INFO:teuthology.orchestra.run.smithi165.stdout:UBUNTU_CODENAME=focal
2022-01-13T07:19:25.137 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server
2022-01-13T07:19:25.156 INFO:teuthology.lock.ops:Updating [smithi189.front.sepia.ceph.com]: set os type and version on server
2022-01-13T07:19:25.156 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com'
2022-01-13T07:19:25.391 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64
2022-01-13T07:19:25.398 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="Ubuntu"
2022-01-13T07:19:25.398 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-01-13T07:19:25.399 INFO:teuthology.orchestra.run.smithi189.stdout:ID=ubuntu
2022-01-13T07:19:25.399 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE=debian
2022-01-13T07:19:25.399 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-01-13T07:19:25.399 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="20.04"
2022-01-13T07:19:25.400 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://www.ubuntu.com/"
2022-01-13T07:19:25.400 INFO:teuthology.orchestra.run.smithi189.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-01-13T07:19:25.400 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-01-13T07:19:25.400 INFO:teuthology.orchestra.run.smithi189.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-01-13T07:19:25.400 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_CODENAME=focal
2022-01-13T07:19:25.401 INFO:teuthology.orchestra.run.smithi189.stdout:UBUNTU_CODENAME=focal
2022-01-13T07:19:25.401 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server
2022-01-13T07:19:25.427 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: set os type and version on server
2022-01-13T07:19:25.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com'
2022-01-13T07:19:25.694 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64
2022-01-13T07:19:25.701 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="Ubuntu"
2022-01-13T07:19:25.701 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-01-13T07:19:25.701 INFO:teuthology.orchestra.run.smithi192.stdout:ID=ubuntu
2022-01-13T07:19:25.702 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE=debian
2022-01-13T07:19:25.702 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-01-13T07:19:25.702 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="20.04"
2022-01-13T07:19:25.702 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.ubuntu.com/"
2022-01-13T07:19:25.703 INFO:teuthology.orchestra.run.smithi192.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-01-13T07:19:25.703 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-01-13T07:19:25.703 INFO:teuthology.orchestra.run.smithi192.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-01-13T07:19:25.703 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_CODENAME=focal
2022-01-13T07:19:25.703 INFO:teuthology.orchestra.run.smithi192.stdout:UBUNTU_CODENAME=focal
2022-01-13T07:19:25.704 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server
2022-01-13T07:19:25.806 INFO:teuthology.dispatcher.supervisor:Running job 6610795
2022-01-13T07:19:25.806 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6fc2011361437a9dfe4e45b50de224392eed8abc/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-01-12_19:45:04-rados-WIP_GBH_NCB_expand_device_fix_1-distro-basic-smithi/6610795 --name benhanokh-2022-01-12_19:45:04-rados-WIP_GBH_NCB_expand_device_fix_1-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2022-01-12_19:45:04-rados-WIP_GBH_NCB_expand_device_fix_1-distro-basic-smithi/6610795/orig.config.yaml
2022-01-13T07:19:25.810 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-01-12_19:45:04-rados-WIP_GBH_NCB_expand_device_fix_1-distro-basic-smithi/6610795
2022-01-13T07:19:25.811 INFO:teuthology.dispatcher.supervisor:Job PID: 587
2022-01-13T07:19:25.811 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-13T19:20:21.259 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-01-13T19:20:21.444 INFO:teuthology.kill:Killing Pids: {587}
2022-01-13T19:20:22.630 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-01-13T19:20:22.631 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-01-13T19:20:22.631 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-01-13T19:20:22.632 INFO:teuthology.misc:Compressing logs...
2022-01-13T19:20:22.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com'
2022-01-13T19:20:22.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com'
2022-01-13T19:20:22.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com'
2022-01-13T19:20:23.328 INFO:teuthology.misc:Compressing logs...
2022-01-13T19:20:24.520 INFO:teuthology.misc:Compressing logs...
2022-01-13T19:22:38.573 INFO:teuthology.kill:No teuthology processes running
2022-01-13T19:22:39.744 INFO:teuthology.kill:Nuking machines: ['smithi165', 'smithi189', 'smithi192']
2022-01-13T19:22:40.718 INFO:teuthology.kill:2022-01-13 19:22:40,717.717 INFO:teuthology.nuke:targets:
2022-01-13T19:22:40.718 INFO:teuthology.kill:  smithi165.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlr42Ek+n8wuz5FrxUMcWN93MvffW6OopKUEUZB6SvYOXfjyMakXzEKH/epeyMfqrEsXXFTGDUQdrHofoxZ8MWSu/xFqeNmzlSmLdn612tHOF/VONFNvjjXNcwVzDqhwxZ+0OCOXWayKRjRhjQEKi7e2BNJGynUKpzkI3w518+7GhCO/NCn+Q+IaWnQYQgVaypOku7JeJ4OTjF41iNu3kcJuueL4cEOu4eowYyPMEJdqTDfJie/yoGzOzX5r/r1+vkS5vPUZ0Yp4Sw/OxqTUgLWN/jTYsli7Fm+oU3GMZeIbfZ89pqPXSEs9Rw6Loj64qnNIa45xeh0zH0DH9T0lXyh5zUtnujQjh+24pj+rH9L9ToY7CAAyy+y8rDQqRNlpsyNW4FZ224m+VCxrFchR8c3OIt9oCnRPJdJevJeVCC49c21djuP3BA8dl2+0FzGbfhHG8kK1Rkd2m646yqJuJtoTrav2A3BbVZWweCQLAgVYhnR1pSuIRwTvS+F5KGtss=
2022-01-13T19:22:40.719 INFO:teuthology.kill:  smithi189.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMGyFRjGRnXmD1j/KvpVRnTifMIN8owo4wnPd2vL61UZ1EPAji9laqoDj+vmdVcag04y+QsY2ymmI/AtPhG9kd5W0EFJWrT0vPXv++7jEUyw2qo7vlc1uus8JCAlKBUvCipQ+H2bbKx+OqtlzNJzydEIsIKTl5YImPFv3/kSSljdLBzQ2cD/owGS4IXUoqWjR6mcIlOQ2dBuZGqrUulLRKeaOj/fCL18/vHp7ge/CsJsKfvnjZrpqk5sOEi0wZFbRaurxmpeKHcwmrej1FCHIa9rZ2kAf9chsFOxtwoH7sIuWmDisb3rjkTqUMthOjs4YLPkSz47fWu7bbLjBno8vXlAMHRwek+ddCu0nJZo81EEa1DHskVWE/reHmYSK+5i+wAUQ/9JSK50lodNJsFxrjWRXN1BA8+HdPdcFyUXakAirrU1+WK82VqFMs8iEPGMFy3JDxUe0nFyXLMmhk7rlS/571GiZMMZT80umNLEF/Oil2FzY+TKTg20bM9jNUrE8=
2022-01-13T19:22:40.719 INFO:teuthology.kill:  smithi192.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChAxAtZueqJAMc+YZFu7VFu0UepckoxHcJ6UiEXHcEcT0j9CivoB6gCK5+8gbh6t6gpKy9BbY4xRPSM5JGOo69Mr8e3TIIxVNvJ3BcVCgpzIarEP7h+3BPVbY6NQHQaXxjY+Dw8oHJ7Au5Yzv2DjYy9cCIGPDLOQ8TAlQ5HxEiRQ1Vm862m0vh029tM1yeTMcSS7YzRvHSjodZmdILkrCrg5CNwH6gSZpQ9AjQqGa9ngkoRCtZiReDQQ3hfzotCPa1HSFYZwIV249K/Nk6EYQ8vZdyoUeVxVJT8VZCxOmZ0+o5OF/k/7jYWBnfakb98KaMH1erGhKRy3KQ7SGdUsuMlYZsgFFtlSAErqT4Xf8qUhhYGmCPk0/avjECzylG3I447W5oTG2YrCX+dFvQjUXHFiKbAzCeUk0IUapvwUw5mqZsDZKT+RFd9QwK8x9A/U/mRZoEkefaCayo7qimtuwwK6V3eKejoAcVk+jC3XGSzPQzKC2Kk11ezE2s8Dr23Bk=
2022-01-13T19:22:40.745 INFO:teuthology.kill:2022-01-13 19:22:40,745.745 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-13T19:22:40.749 INFO:teuthology.kill:2022-01-13 19:22:40,749.749 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-13T19:22:40.750 INFO:teuthology.kill:2022-01-13 19:22:40,750.750 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-13T19:22:40.827 INFO:teuthology.kill:2022-01-13 19:22:40,827.827 INFO:teuthology.orchestra.console:Power off smithi192
2022-01-13T19:22:40.840 INFO:teuthology.kill:2022-01-13 19:22:40,840.840 INFO:teuthology.orchestra.console:Power off smithi165
2022-01-13T19:22:40.851 INFO:teuthology.kill:2022-01-13 19:22:40,851.851 INFO:teuthology.orchestra.console:Power off smithi189
2022-01-13T19:22:49.214 INFO:teuthology.kill:2022-01-13 19:22:49,214.214 INFO:teuthology.orchestra.console:Power off for smithi165 completed
2022-01-13T19:22:49.359 INFO:teuthology.kill:2022-01-13 19:22:49,358.358 INFO:teuthology.lock.ops:unlocked: smithi165.front.sepia.ceph.com
2022-01-13T19:22:53.321 INFO:teuthology.kill:2022-01-13 19:22:53,320.320 INFO:teuthology.orchestra.console:Power off for smithi192 completed
2022-01-13T19:22:53.331 INFO:teuthology.kill:2022-01-13 19:22:53,330.330 INFO:teuthology.orchestra.console:Power off for smithi189 completed
2022-01-13T19:22:53.466 INFO:teuthology.kill:2022-01-13 19:22:53,465.465 INFO:teuthology.lock.ops:unlocked: smithi192.front.sepia.ceph.com
2022-01-13T19:22:53.472 INFO:teuthology.kill:2022-01-13 19:22:53,471.471 INFO:teuthology.lock.ops:unlocked: smithi189.front.sepia.ceph.com
2022-01-13T19:24:53.800 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-13T19:24:53.920 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi165 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-13_14:57:55-rados-wip-yuri5-testing-2022-01-12-1534-distro-default-smithi/6613082
2022-01-13T19:24:53.920 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi189 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-13_14:57:55-rados-wip-yuri5-testing-2022-01-12-1534-distro-default-smithi/6613081
2022-01-13T19:24:53.921 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi192 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-13_14:57:55-rados-wip-yuri5-testing-2022-01-12-1534-distro-default-smithi/6613082