2022-03-26T16:35:54.190 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-26T16:35:54.317 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi065.front.sepia.ceph.com' reimaging 2022-03-26T16:35:54.318 INFO:teuthology.lock.ops:Updating [ubuntu@smithi065.front.sepia.ceph.com]: reset os type and version on server 2022-03-26T16:35:54.318 INFO:teuthology.lock.ops:Updating smithi065.front.sepia.ceph.com on lock server 2022-03-26T16:35:54.336 INFO:teuthology.lock.ops:Node 'ubuntu@smithi065.front.sepia.ceph.com' reimaging is complete 2022-03-26T16:35:54.336 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi117.front.sepia.ceph.com' reimaging 2022-03-26T16:35:54.336 INFO:teuthology.lock.ops:Updating [ubuntu@smithi117.front.sepia.ceph.com]: reset os type and version on server 2022-03-26T16:35:54.337 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2022-03-26T16:35:54.357 INFO:teuthology.lock.ops:Node 'ubuntu@smithi117.front.sepia.ceph.com' reimaging is complete 2022-03-26T16:35:54.565 INFO:teuthology.provision.fog.smithi065:Scheduling deploy of ubuntu 20.04 2022-03-26T16:35:54.600 INFO:teuthology.provision.fog.smithi117:Scheduling deploy of ubuntu 20.04 2022-03-26T16:35:54.954 INFO:teuthology.orchestra.console:Power off smithi065 2022-03-26T16:35:55.008 INFO:teuthology.orchestra.console:Power off smithi117 2022-03-26T16:36:03.324 INFO:teuthology.orchestra.console:Power off for smithi065 completed 2022-03-26T16:36:03.362 INFO:teuthology.orchestra.console:Power off for smithi117 completed 2022-03-26T16:36:03.426 INFO:teuthology.orchestra.console:Power on smithi065 2022-03-26T16:36:03.464 INFO:teuthology.orchestra.console:Power on smithi117 2022-03-26T16:36:11.830 INFO:teuthology.orchestra.console:Power on for smithi117 completed 2022-03-26T16:36:11.897 INFO:teuthology.orchestra.console:Power on for smithi065 completed 2022-03-26T16:36:11.931 INFO:teuthology.provision.fog.smithi117:Waiting for deploy to finish 2022-03-26T16:36:11.998 INFO:teuthology.provision.fog.smithi065:Waiting for deploy to finish 2022-03-26T16:41:12.402 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-26T16:41:12.842 INFO:teuthology.provision.fog.smithi117:Node is ready 2022-03-26T16:41:12.849 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117.front.sepia.ceph.com 2022-03-26T16:41:12.858 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117 smithi117.front.sepia.ceph.com smithi117 2022-03-26T16:41:13.020 INFO:teuthology.provision.fog.smithi117:Deploy complete! 2022-03-26T16:41:21.294 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-26T16:41:21.723 INFO:teuthology.provision.fog.smithi065:Node is ready 2022-03-26T16:41:21.729 INFO:teuthology.orchestra.run.smithi065.stdout:smithi065.front.sepia.ceph.com 2022-03-26T16:41:21.780 INFO:teuthology.orchestra.run.smithi065.stdout:172.21.15.65 smithi065.front.sepia.ceph.com smithi065 2022-03-26T16:41:21.954 INFO:teuthology.provision.fog.smithi065:Deploy complete! 2022-03-26T16:41:22.136 INFO:teuthology.lock.ops:Checking smithi065.front.sepia.ceph.com 2022-03-26T16:41:22.137 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-26T16:41:22.155 INFO:teuthology.lock.ops:Checking smithi117.front.sepia.ceph.com 2022-03-26T16:41:22.156 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-26T16:41:22.174 INFO:teuthology.lock.ops:Updating [smithi065.front.sepia.ceph.com]: set os type and version on server 2022-03-26T16:41:22.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi065.front.sepia.ceph.com' 2022-03-26T16:41:22.362 INFO:teuthology.orchestra.run.smithi065.stdout:x86_64 2022-03-26T16:41:22.369 INFO:teuthology.orchestra.run.smithi065.stdout:NAME="Ubuntu" 2022-03-26T16:41:22.370 INFO:teuthology.orchestra.run.smithi065.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-26T16:41:22.370 INFO:teuthology.orchestra.run.smithi065.stdout:ID=ubuntu 2022-03-26T16:41:22.370 INFO:teuthology.orchestra.run.smithi065.stdout:ID_LIKE=debian 2022-03-26T16:41:22.371 INFO:teuthology.orchestra.run.smithi065.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-26T16:41:22.371 INFO:teuthology.orchestra.run.smithi065.stdout:VERSION_ID="20.04" 2022-03-26T16:41:22.371 INFO:teuthology.orchestra.run.smithi065.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-26T16:41:22.371 INFO:teuthology.orchestra.run.smithi065.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-26T16:41:22.371 INFO:teuthology.orchestra.run.smithi065.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-26T16:41:22.372 INFO:teuthology.orchestra.run.smithi065.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-26T16:41:22.372 INFO:teuthology.orchestra.run.smithi065.stdout:VERSION_CODENAME=focal 2022-03-26T16:41:22.372 INFO:teuthology.orchestra.run.smithi065.stdout:UBUNTU_CODENAME=focal 2022-03-26T16:41:22.373 INFO:teuthology.lock.ops:Updating smithi065.front.sepia.ceph.com on lock server 2022-03-26T16:41:22.391 INFO:teuthology.lock.ops:Updating [smithi117.front.sepia.ceph.com]: set os type and version on server 2022-03-26T16:41:22.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2022-03-26T16:41:22.562 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2022-03-26T16:41:22.568 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Ubuntu" 2022-03-26T16:41:22.569 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-26T16:41:22.569 INFO:teuthology.orchestra.run.smithi117.stdout:ID=ubuntu 2022-03-26T16:41:22.569 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE=debian 2022-03-26T16:41:22.569 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-26T16:41:22.570 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="20.04" 2022-03-26T16:41:22.570 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-26T16:41:22.570 INFO:teuthology.orchestra.run.smithi117.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-26T16:41:22.570 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-26T16:41:22.571 INFO:teuthology.orchestra.run.smithi117.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-26T16:41:22.571 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_CODENAME=focal 2022-03-26T16:41:22.571 INFO:teuthology.orchestra.run.smithi117.stdout:UBUNTU_CODENAME=focal 2022-03-26T16:41:22.571 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2022-03-26T16:41:22.663 INFO:teuthology.dispatcher.supervisor:Running job 6757489 2022-03-26T16:41:22.663 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757489 --name vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} -- /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757489/orig.config.yaml 2022-03-26T16:41:22.667 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757489 2022-03-26T16:41:22.668 INFO:teuthology.dispatcher.supervisor:Job PID: 30529 2022-03-26T16:41:22.668 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-26T23:11:50.343 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-26T23:11:50.527 INFO:teuthology.kill:Killing Pids: {30529} 2022-03-26T23:11:51.690 INFO:teuthology.task.internal:roles: ubuntu@smithi065.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-26T23:11:51.691 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-26T23:11:51.691 INFO:teuthology.misc:Compressing logs... 2022-03-26T23:11:51.692 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi065.front.sepia.ceph.com' 2022-03-26T23:11:52.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2022-03-26T23:11:53.060 INFO:teuthology.misc:Compressing logs... 2022-03-26T23:11:53.312 INFO:teuthology.misc:Compressing logs... 2022-03-26T23:22:16.241 INFO:teuthology.kill:No teuthology processes running 2022-03-26T23:22:17.394 INFO:teuthology.kill:Nuking machines: ['smithi065', 'smithi117'] 2022-03-26T23:22:18.337 INFO:teuthology.kill:2022-03-26 23:22:18,337.337 INFO:teuthology.nuke:targets: 2022-03-26T23:22:18.338 INFO:teuthology.kill: smithi065.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy1/I0J8WtyGyVJCXHoQUxJ7u2B5S0albhE0DfY8yftC5/9YxzLzCBzt1m17yy/HoNKDHC7MvgJ2ew0bVHKFV+19bROPN9bvQ40HxaF/WHfiwVYoKIZon42mHhiNgnaSkABsRkx4vEm2bynahYn2wM1jKgBegDjGLGshGu6+Mf8h4b7dikaQWdwedqBrJhrV/6YPOtyEQx+Bnx8gVoDpAgOeqh+vWGuA9VVPSU8NfbntnDf3KJAB4XlZCxGz6nBfmxOkj7U5K4WmA7m09w1Pcjk7oFSBKCxF3KWWJHihcRTro6JVwgkaIbppLAR+Ny90Fl8ADyaNgIC0KuRztc6FxhT+tu+1ROGk2zt5q+jWcjCPR+JO9cMDHdRf3rmw5tucgtJ0RkRTf0xbfl66X+b8jhrfGr5//96paNi9nBT2BJeOpR+OS4U79y6ixd38fhUxRL4QGWw97IYNch6I7iwqaKp+mwc3ixfb/8RuGXHNkb5XIFbzqMf2kQ0sbbPPGaYn8= 2022-03-26T23:22:18.338 INFO:teuthology.kill: smithi117.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcA2sgczksWnJJ9fhhMvse2inkszk8hIh9NWGy7bWWxNSeNHqikK4zJ8YDSasL3snI8vMZW9IBHypEwz9XsjsmVFOwp/3VSyEDCobaQgS2b43Ak1IWtATjmj4MZu5lB8ZGI1DwVb5PXwh/jNBtccAd1vrJ1SDtOUjTZL3CVCAQUn8Bj1E1LPvGjnFVBH09K8CnxCCMKipH0fw1eN1OFKv59e6oDq9MzqiK37OwP7OLjZsTMlIu5rIIGPPXbC8Rj8cXCexVU1UpOvFitvZZeEdDHjqCW27qAO5ACnSQRca6jCQSmCHJM0++9dum+hQ6N2Z3JhXou0G7Wdr+zP0DnQTlQrXLuWz7pIxzetUTqNe0OpMUv0Rb/s7UG2lSJdnga0/XnGdwh0YYz1UeKjsk0o7c6wf/NgcAcXiJ1xWuonzN445VWEyIM7n+QBncpUYJZQHjA9yl1nTil2t4/yWu9yd4hROTZ/QMBqb2ld0PU4+OYq7nYE3NbRnbaDfjWVf2nBE= 2022-03-26T23:22:18.363 INFO:teuthology.kill:2022-03-26 23:22:18,363.363 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-26T23:22:18.366 INFO:teuthology.kill:2022-03-26 23:22:18,366.366 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-26T23:22:18.440 INFO:teuthology.kill:2022-03-26 23:22:18,439.439 INFO:teuthology.orchestra.console:Power off smithi065 2022-03-26T23:22:18.453 INFO:teuthology.kill:2022-03-26 23:22:18,453.453 INFO:teuthology.orchestra.console:Power off smithi117 2022-03-26T23:22:30.922 INFO:teuthology.kill:2022-03-26 23:22:30,921.921 INFO:teuthology.orchestra.console:Power off for smithi065 completed 2022-03-26T23:22:30.925 INFO:teuthology.kill:2022-03-26 23:22:30,925.925 INFO:teuthology.orchestra.console:Power off for smithi117 completed 2022-03-26T23:22:31.081 INFO:teuthology.kill:2022-03-26 23:22:31,080.080 INFO:teuthology.lock.ops:unlocked: smithi117.front.sepia.ceph.com 2022-03-26T23:22:31.084 INFO:teuthology.kill:2022-03-26 23:22:31,084.084 INFO:teuthology.lock.ops:unlocked: smithi065.front.sepia.ceph.com 2022-03-26T23:24:31.400 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-26T23:24:31.512 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi065 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757581 2022-03-26T23:24:31.512 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi117 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757581