2022-02-23T13:26:49.932 INFO:root:teuthology version: 1.1.0-eea25212
2022-02-23T13:26:50.100 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging
2022-02-23T13:26:50.102 INFO:teuthology.lock.ops:Updating [ubuntu@smithi068.front.sepia.ceph.com]: reset os type and version on server
2022-02-23T13:26:50.102 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server
2022-02-23T13:26:50.129 INFO:teuthology.lock.ops:Node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging is complete
2022-02-23T13:26:50.129 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi100.front.sepia.ceph.com' reimaging
2022-02-23T13:26:50.130 INFO:teuthology.lock.ops:Updating [ubuntu@smithi100.front.sepia.ceph.com]: reset os type and version on server
2022-02-23T13:26:50.131 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server
2022-02-23T13:26:50.156 INFO:teuthology.lock.ops:Node 'ubuntu@smithi100.front.sepia.ceph.com' reimaging is complete
2022-02-23T13:26:50.157 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging
2022-02-23T13:26:50.158 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server
2022-02-23T13:26:50.158 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2022-02-23T13:26:50.178 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete
2022-02-23T13:26:50.392 INFO:teuthology.provision.fog.smithi068:Scheduling deploy of ubuntu 20.04
2022-02-23T13:26:50.411 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of ubuntu 20.04
2022-02-23T13:26:50.426 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of ubuntu 20.04
2022-02-23T13:26:50.797 INFO:teuthology.orchestra.console:Power off smithi068
2022-02-23T13:26:50.815 INFO:teuthology.orchestra.console:Power off smithi100
2022-02-23T13:26:50.885 INFO:teuthology.orchestra.console:Power off smithi116
2022-02-23T13:26:59.185 INFO:teuthology.orchestra.console:Power off for smithi100 completed
2022-02-23T13:26:59.250 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2022-02-23T13:26:59.287 INFO:teuthology.orchestra.console:Power on smithi100
2022-02-23T13:26:59.351 INFO:teuthology.orchestra.console:Power on smithi116
2022-02-23T13:27:03.295 INFO:teuthology.orchestra.console:Power off for smithi068 completed
2022-02-23T13:27:03.397 INFO:teuthology.orchestra.console:Power on smithi068
2022-02-23T13:27:03.614 INFO:teuthology.orchestra.console:Power on for smithi116 completed
2022-02-23T13:27:03.716 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish
2022-02-23T13:27:07.652 INFO:teuthology.orchestra.console:Power on for smithi068 completed
2022-02-23T13:27:07.711 INFO:teuthology.orchestra.console:Power on for smithi100 completed
2022-02-23T13:27:07.757 INFO:teuthology.provision.fog.smithi068:Waiting for deploy to finish
2022-02-23T13:27:07.813 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish
2022-02-23T13:32:02.218 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-23T13:32:02.771 INFO:teuthology.provision.fog.smithi100:Node is ready
2022-02-23T13:32:02.778 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com
2022-02-23T13:32:02.826 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100
2022-02-23T13:32:03.001 INFO:teuthology.provision.fog.smithi100:Deploy complete!
2022-02-23T13:32:12.375 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-23T13:32:12.845 INFO:teuthology.provision.fog.smithi116:Node is ready
2022-02-23T13:32:12.852 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com
2022-02-23T13:32:12.859 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116
2022-02-23T13:32:13.050 INFO:teuthology.provision.fog.smithi116:Deploy complete!
2022-02-23T13:32:15.158 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-23T13:32:15.621 INFO:teuthology.provision.fog.smithi068:Node is ready
2022-02-23T13:32:15.628 INFO:teuthology.orchestra.run.smithi068.stdout:smithi068.front.sepia.ceph.com
2022-02-23T13:32:15.635 INFO:teuthology.orchestra.run.smithi068.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068
2022-02-23T13:32:15.850 INFO:teuthology.provision.fog.smithi068:Deploy complete!
2022-02-23T13:32:16.103 INFO:teuthology.lock.ops:Checking smithi068.front.sepia.ceph.com
2022-02-23T13:32:16.105 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-23T13:32:16.124 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com
2022-02-23T13:32:16.125 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-23T13:32:16.142 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com
2022-02-23T13:32:16.142 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-23T13:32:16.160 INFO:teuthology.lock.ops:Updating [smithi068.front.sepia.ceph.com]: set os type and version on server
2022-02-23T13:32:16.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com'
2022-02-23T13:32:16.342 INFO:teuthology.orchestra.run.smithi068.stdout:x86_64
2022-02-23T13:32:16.349 INFO:teuthology.orchestra.run.smithi068.stdout:NAME="Ubuntu"
2022-02-23T13:32:16.349 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-23T13:32:16.349 INFO:teuthology.orchestra.run.smithi068.stdout:ID=ubuntu
2022-02-23T13:32:16.349 INFO:teuthology.orchestra.run.smithi068.stdout:ID_LIKE=debian
2022-02-23T13:32:16.350 INFO:teuthology.orchestra.run.smithi068.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-23T13:32:16.350 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_ID="20.04"
2022-02-23T13:32:16.350 INFO:teuthology.orchestra.run.smithi068.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-23T13:32:16.350 INFO:teuthology.orchestra.run.smithi068.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-23T13:32:16.350 INFO:teuthology.orchestra.run.smithi068.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-23T13:32:16.351 INFO:teuthology.orchestra.run.smithi068.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-23T13:32:16.351 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_CODENAME=focal
2022-02-23T13:32:16.352 INFO:teuthology.orchestra.run.smithi068.stdout:UBUNTU_CODENAME=focal
2022-02-23T13:32:16.353 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server
2022-02-23T13:32:16.374 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server
2022-02-23T13:32:16.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com'
2022-02-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64
2022-02-23T13:32:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu"
2022-02-23T13:32:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-23T13:32:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu
2022-02-23T13:32:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian
2022-02-23T13:32:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-23T13:32:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="20.04"
2022-02-23T13:32:16.696 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-23T13:32:16.696 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-23T13:32:16.697 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-23T13:32:16.697 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-23T13:32:16.698 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=focal
2022-02-23T13:32:16.698 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=focal
2022-02-23T13:32:16.699 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server
2022-02-23T13:32:16.720 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server
2022-02-23T13:32:16.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com'
2022-02-23T13:32:16.947 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64
2022-02-23T13:32:16.955 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Ubuntu"
2022-02-23T13:32:16.955 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-23T13:32:16.955 INFO:teuthology.orchestra.run.smithi116.stdout:ID=ubuntu
2022-02-23T13:32:16.956 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE=debian
2022-02-23T13:32:16.956 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-23T13:32:16.956 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="20.04"
2022-02-23T13:32:16.956 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-23T13:32:16.957 INFO:teuthology.orchestra.run.smithi116.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-23T13:32:16.957 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-23T13:32:16.957 INFO:teuthology.orchestra.run.smithi116.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-23T13:32:16.957 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_CODENAME=focal
2022-02-23T13:32:16.957 INFO:teuthology.orchestra.run.smithi116.stdout:UBUNTU_CODENAME=focal
2022-02-23T13:32:16.958 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2022-02-23T13:32:17.056 INFO:teuthology.dispatcher.supervisor:Running job 6702281
2022-02-23T13:32:17.057 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-23_13:21:14-rados-GBH_safe_shutdown_v2_basecode_sanity_check_disabled-distro-basic-smithi/6702281 --name benhanokh-2022-02-23_13:21:14-rados-GBH_safe_shutdown_v2_basecode_sanity_check_disabled-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2022-02-23_13:21:14-rados-GBH_safe_shutdown_v2_basecode_sanity_check_disabled-distro-basic-smithi/6702281/orig.config.yaml
2022-02-23T13:32:17.061 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-23_13:21:14-rados-GBH_safe_shutdown_v2_basecode_sanity_check_disabled-distro-basic-smithi/6702281
2022-02-23T13:32:17.061 INFO:teuthology.dispatcher.supervisor:Job PID: 25378
2022-02-23T13:32:17.062 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-23T20:02:45.768 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-23T20:02:45.943 INFO:teuthology.kill:Killing Pids: {25378}
2022-02-23T20:02:47.258 INFO:teuthology.task.internal:roles: ubuntu@smithi068.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-02-23T20:02:47.259 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-02-23T20:02:47.260 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-02-23T20:02:47.260 INFO:teuthology.misc:Compressing logs...
2022-02-23T20:02:47.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com'
2022-02-23T20:02:47.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com'
2022-02-23T20:02:47.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com'
2022-02-23T20:02:47.981 INFO:teuthology.misc:Compressing logs...
2022-02-23T20:02:48.662 INFO:teuthology.misc:Compressing logs...
2022-02-23T20:07:34.177 INFO:teuthology.kill:No teuthology processes running
2022-02-23T20:07:35.360 INFO:teuthology.kill:Nuking machines: ['smithi068', 'smithi100', 'smithi116']
2022-02-23T20:07:36.389 INFO:teuthology.kill:2022-02-23 20:07:36,389.389 INFO:teuthology.nuke:targets:
2022-02-23T20:07:36.390 INFO:teuthology.kill:  smithi068.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjb+CBnwmtC45SKwfnCOhI5W+RSIHRglEPyG/32Exhc68iFL6du0bbXJjkudw0SU59zo/+bHzZ9LtCeU61q7yNn1WiY6C60tk0kIrmRVuMSHCVSnL95Eq8ep0b2fOIAJLNdhdnebhI+5qd1239gAP36pVEcYQVv2N3dm2tMdSIOTdL3IKzmioKK4Lw9v73CNV/dHiUrjUt0vRdAQK88+WPtTCg1r5UcigZWdfmweUb63jxL1N8xHjTbkATrrl2tSIeyywXAY5GEsf4NhK8dmW2mzYBqcBzIa3z8V/hY/i/jYfAi4isSqPJpNgXVMLaXE5hfUuxFHuxaebYKGDiG66KvamL3vuaoTDI53aBZ/39kxAXMeF0RkrIOJQ2Olz5wUG7ijBVLrRcg0dX77ZyCoK9PGzbktON5Z+0Q0AoysfVcufw5YSwhOvbR9XtZx1xQuY2bR3qgBOKu3rY2IESPDMeD7KraVn0sOBqx04I7jUFQXMX3IOSVM04g9TLMqHFogc=
2022-02-23T20:07:36.390 INFO:teuthology.kill:  smithi100.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1xO7arTmk8lDI95Mi6CDq8yahDcb6z2wzU0QGlj2CId1AqDFGyh/rqTkM7zD3SZARei4edJ17xBfTjNGyFetvLoTGJzLzQu/ZCaD5gQFOUkZmUvTU35c0rm72gKUMUZgb3WY9epMroUcgFbx6igwxHQMSjyMFL1AsqrI8cuN5xhjgLIbPaMhIx+cGTJt8cVxcpGb4pV0lrCthVwse5eFfEQsuvVHbSWdfcOfs0xH2YLYVFwyWUluerwLegGT4dTlxc1W8krbmc5+gDv8caht+4bVG0QgwDS4LJtvV29PTF03GsARttiG6m1V/g6MvPpEdLQMkG9NlhvIwYUuvm7Nf0oPa9B7rd+tL0Vuf7YYiv0AJ8mBx90pe6S8t6y9kq8vmy1f+rEZ2E9AB23EJnuChooLPnjRbucJqC+hTyIn2NA5hqwxBfTkdwIDsGz3sJVxM4HGypdon9c4FVLBVLT0cy8OW93O3QPibzDYot9x5vhbIJlw2qqhRJ7YGJZu/qB0=
2022-02-23T20:07:36.390 INFO:teuthology.kill:  smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/C3ZqDg8ikYcJqwFbzTOoe+Xz4fudVUS8ryfi1b/jaR7h0fXxJxTtzyj46Kh2AR5l+T73tyVki6C+LiAW2ud7JHb34FtjUgbp3MvujpjwHqRduXuvcIFfzFtYIPSvKwbrT6LX5ThZPnsRcM6iaF1BoTPJGZAz4thLPw5zVs4vb36qwFj2f6uhjzi9zKQE1ORSqShmO/7o/xNR5d8+tKHa4Ct+/t4YNsfrthbNr1G+dP1Xt4H3aLWhbT3YtdhCL3tCMXV/Fu52bNrWEDBkK+B/M9DuFfVWAqDuUMEp1MU6zeOeuXvwtUxHMMSqSeaxtpP15ZfYqRBABuUnqmWKjyeRJSkzhKskmoBsJ6DgtrDNYHfhunQ8FRNjYEWIggw0i/z/0WWZgmqyaO58iXEhW0drtffqr/AG4ucFK86TpX34AJSF/HBgboP3kGFjoFXk1AQxuwXKIrzE2w+IHedmKXftbbPIf0bdfLWBEWW/btXCXn17K4gDAaUPU2mGWBgIrL8=
2022-02-23T20:07:36.420 INFO:teuthology.kill:2022-02-23 20:07:36,420.420 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-23T20:07:36.423 INFO:teuthology.kill:2022-02-23 20:07:36,423.423 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-23T20:07:36.425 INFO:teuthology.kill:2022-02-23 20:07:36,425.425 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-23T20:07:36.517 INFO:teuthology.kill:2022-02-23 20:07:36,516.516 INFO:teuthology.orchestra.console:Power off smithi068
2022-02-23T20:07:36.534 INFO:teuthology.kill:2022-02-23 20:07:36,533.533 INFO:teuthology.orchestra.console:Power off smithi116
2022-02-23T20:07:36.546 INFO:teuthology.kill:2022-02-23 20:07:36,546.546 INFO:teuthology.orchestra.console:Power off smithi100
2022-02-23T20:07:44.911 INFO:teuthology.kill:2022-02-23 20:07:44,910.910 INFO:teuthology.orchestra.console:Power off for smithi068 completed
2022-02-23T20:07:45.060 INFO:teuthology.kill:2022-02-23 20:07:45,059.059 INFO:teuthology.lock.ops:unlocked: smithi068.front.sepia.ceph.com
2022-02-23T20:07:49.019 INFO:teuthology.kill:2022-02-23 20:07:49,018.018 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2022-02-23T20:07:49.024 INFO:teuthology.kill:2022-02-23 20:07:49,024.024 INFO:teuthology.orchestra.console:Power off for smithi100 completed
2022-02-23T20:07:49.165 INFO:teuthology.kill:2022-02-23 20:07:49,164.164 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com
2022-02-23T20:07:49.168 INFO:teuthology.kill:2022-02-23 20:07:49,168.168 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com
2022-02-23T20:09:49.485 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-23T20:09:49.611 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi068 but it was locked by another job: /home/teuthworker/archive/nojha-2022-02-23_17:58:44-rados-GBH_safe_shutdown_v2_basecode_sanity_check_disabled_2-distro-basic-smithi/6702926
2022-02-23T20:09:49.611 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi100 but it was locked by another job: /home/teuthworker/archive/nojha-2022-02-23_17:58:44-rados-GBH_safe_shutdown_v2_basecode_sanity_check_disabled_2-distro-basic-smithi/6702925
2022-02-23T20:09:49.612 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi116 but it was locked by another job: /home/teuthworker/archive/nojha-2022-02-23_17:58:44-rados-GBH_safe_shutdown_v2_basecode_sanity_check_disabled_2-distro-basic-smithi/6702926