2021-11-10T22:30:36.473 INFO:root:teuthology version: 1.1.0-91815aeb 2021-11-10T22:30:36.598 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi137.front.sepia.ceph.com' reimaging 2021-11-10T22:30:36.599 INFO:teuthology.lock.ops:Updating [ubuntu@smithi137.front.sepia.ceph.com]: reset os type and version on server 2021-11-10T22:30:36.599 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2021-11-10T22:30:36.618 INFO:teuthology.lock.ops:Node 'ubuntu@smithi137.front.sepia.ceph.com' reimaging is complete 2021-11-10T22:30:36.619 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging 2021-11-10T22:30:36.619 INFO:teuthology.lock.ops:Updating [ubuntu@smithi155.front.sepia.ceph.com]: reset os type and version on server 2021-11-10T22:30:36.619 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2021-11-10T22:30:36.640 INFO:teuthology.lock.ops:Node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging is complete 2021-11-10T22:30:36.864 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of ubuntu 18.04 2021-11-10T22:30:36.869 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of ubuntu 18.04 2021-11-10T22:30:37.295 INFO:teuthology.orchestra.console:Power off smithi137 2021-11-10T22:30:37.313 INFO:teuthology.orchestra.console:Power off smithi155 2021-11-10T22:30:37.457 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2021-11-10T22:30:37.468 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2021-11-10T22:30:37.558 INFO:teuthology.orchestra.console:Power on smithi137 2021-11-10T22:30:37.569 INFO:teuthology.orchestra.console:Power on smithi155 2021-11-10T22:30:45.936 INFO:teuthology.orchestra.console:Power on for smithi155 completed 2021-11-10T22:30:46.038 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2021-11-10T22:30:46.047 INFO:teuthology.orchestra.console:Power on for smithi137 completed 2021-11-10T22:30:46.148 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish 2021-11-10T22:35:46.137 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-11-10T22:35:52.153 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-11-10T22:35:58.263 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-10T22:35:58.512 INFO:teuthology.provision.fog.smithi155:Node is ready 2021-11-10T22:35:58.559 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2021-11-10T22:35:58.608 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2021-11-10T22:35:58.898 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2021-11-10T22:36:05.747 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-10T22:36:05.988 INFO:teuthology.provision.fog.smithi137:Node is ready 2021-11-10T22:36:06.036 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com 2021-11-10T22:36:06.085 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137 2021-11-10T22:36:06.352 INFO:teuthology.provision.fog.smithi137:Deploy complete! 2021-11-10T22:36:06.523 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com 2021-11-10T22:36:06.524 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-10T22:36:06.543 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2021-11-10T22:36:06.544 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-10T22:36:06.563 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server 2021-11-10T22:36:06.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2021-11-10T22:36:06.784 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2021-11-10T22:36:06.832 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Ubuntu" 2021-11-10T22:36:06.832 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2021-11-10T22:36:06.832 INFO:teuthology.orchestra.run.smithi137.stdout:ID=ubuntu 2021-11-10T22:36:06.832 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE=debian 2021-11-10T22:36:06.833 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2021-11-10T22:36:06.833 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="18.04" 2021-11-10T22:36:06.833 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-10T22:36:06.833 INFO:teuthology.orchestra.run.smithi137.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-10T22:36:06.834 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-10T22:36:06.834 INFO:teuthology.orchestra.run.smithi137.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-10T22:36:06.834 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_CODENAME=bionic 2021-11-10T22:36:06.834 INFO:teuthology.orchestra.run.smithi137.stdout:UBUNTU_CODENAME=bionic 2021-11-10T22:36:06.835 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2021-11-10T22:36:06.854 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2021-11-10T22:36:06.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2021-11-10T22:36:07.091 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2021-11-10T22:36:07.139 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Ubuntu" 2021-11-10T22:36:07.139 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2021-11-10T22:36:07.139 INFO:teuthology.orchestra.run.smithi155.stdout:ID=ubuntu 2021-11-10T22:36:07.140 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE=debian 2021-11-10T22:36:07.140 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2021-11-10T22:36:07.140 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="18.04" 2021-11-10T22:36:07.140 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-10T22:36:07.141 INFO:teuthology.orchestra.run.smithi155.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-10T22:36:07.141 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-10T22:36:07.141 INFO:teuthology.orchestra.run.smithi155.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-10T22:36:07.141 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_CODENAME=bionic 2021-11-10T22:36:07.142 INFO:teuthology.orchestra.run.smithi155.stdout:UBUNTU_CODENAME=bionic 2021-11-10T22:36:07.142 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2021-11-10T22:36:07.239 INFO:teuthology.dispatcher.supervisor:Running job 6496573 2021-11-10T22:36:07.239 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_91815aeb94449f1ea4365867a0fd4c60103057e1/virtualenv/bin/teuthology -v --owner scheduled_ifed01@teuthology --archive /home/teuthworker/archive/ifed01-2021-11-10_19:13:41-rados-wip-ifed-testing1-distro-default-smithi/6496573 --name ifed01-2021-11-10_19:13:41-rados-wip-ifed-testing1-distro-default-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_18.04} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} -- /home/teuthworker/archive/ifed01-2021-11-10_19:13:41-rados-wip-ifed-testing1-distro-default-smithi/6496573/orig.config.yaml 2021-11-10T22:36:07.243 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ifed01-2021-11-10_19:13:41-rados-wip-ifed-testing1-distro-default-smithi/6496573 2021-11-10T22:36:07.244 INFO:teuthology.dispatcher.supervisor:Job PID: 11714 2021-11-10T22:36:07.244 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-11T10:37:01.173 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-11T10:37:01.395 INFO:teuthology.kill:Killing Pids: {11714} 2021-11-11T10:37:02.556 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-11-11T10:37:02.557 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-11-11T10:37:02.558 INFO:teuthology.misc:Compressing logs... 2021-11-11T10:37:02.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2021-11-11T10:37:02.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2021-11-11T10:37:33.180 INFO:teuthology.kill:No teuthology processes running 2021-11-11T10:37:34.334 INFO:teuthology.kill:Nuking machines: ['smithi137', 'smithi155'] 2021-11-11T10:37:35.288 INFO:teuthology.kill:2021-11-11 10:37:35,287.287 INFO:teuthology.nuke:targets: 2021-11-11T10:37:35.288 INFO:teuthology.kill: smithi137.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkHc5GoEWN2A6vnM7aIompJOW4Gon+ADIvhbSaug5B+q7j9GZD+cFtP5yQvatbaS7hXt86d3i1jJsfqKAuz0dmipQTwDh+qI7rCFohc7hS7kETR+mNtehgGlTiBiOmmr9iXYHn7PiYkbm4G/eo6599ywEu9WonF6oSZ7gPXft28EYGDoCKTm78/05ROcrCNHtgfH4RoeRjRZI645KMIC2McWc/jZhm9A5r8L8IjP4c7SDqK9RlS46kmYi9b/LY3QTJewDeL4PS/FWsgmsX27FhsiJhPAvtg6jBC13ojA23uhcD3AhHicwQvXTzY8oq0QGBt82D4/seMVumSbXw6f1L 2021-11-11T10:37:35.289 INFO:teuthology.kill: smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4O45K8SykkXf+D+oqZkOjWmx/j6HT6FGFFUdLqUmtmqgWqldluAldK1100sxHYB8V6k7wRw5FTxGt92oouSuJf6IsAc3POa+JaJdzOjzKflBYYsmp0UySjqO/q1VjL6IKd6m/5v2IquYRpGBBmrqEclZBzISy8ctX9AU61HoFcgOsFHPp86/skjYL6l/XnhAZiQn87iVS2me+Zrjq5RA52hdyqyB9A/R5/KmdKPzJ0liJFpomAI8GIGwiVVDDav7kVTX9yTVHFM6kPBpzUogZkoHjWbM2xz52niVFrVjT7tqrQH4hUZ2PnV2aMiiap5Bq3Z4jzxReW6C5tWm6t54J 2021-11-11T10:37:35.314 INFO:teuthology.kill:2021-11-11 10:37:35,314.314 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-11T10:37:35.319 INFO:teuthology.kill:2021-11-11 10:37:35,319.319 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-11T10:37:35.384 INFO:teuthology.kill:2021-11-11 10:37:35,384.384 INFO:teuthology.orchestra.console:Power off smithi155 2021-11-11T10:37:35.397 INFO:teuthology.kill:2021-11-11 10:37:35,397.397 INFO:teuthology.orchestra.console:Power off smithi137 2021-11-11T10:37:43.757 INFO:teuthology.kill:2021-11-11 10:37:43,756.756 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2021-11-11T10:37:43.762 INFO:teuthology.kill:2021-11-11 10:37:43,762.762 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2021-11-11T10:37:43.904 INFO:teuthology.kill:2021-11-11 10:37:43,903.903 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com 2021-11-11T10:37:43.908 INFO:teuthology.kill:2021-11-11 10:37:43,908.908 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com 2021-11-11T10:39:44.196 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-11T10:39:44.306 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi137 but it was locked by another job: /home/teuthworker/archive/benhanokh-2021-11-10_07:27:45-rados-restore_alloc_file-distro-basic-smithi/6495485 2021-11-11T10:39:44.306 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi155 but it was locked by another job: /home/teuthworker/archive/benhanokh-2021-11-10_07:27:45-rados-restore_alloc_file-distro-basic-smithi/6495485