2022-03-26T23:52:07.714 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-26T23:52:07.843 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi122.front.sepia.ceph.com' reimaging 2022-03-26T23:52:07.844 INFO:teuthology.lock.ops:Updating [ubuntu@smithi122.front.sepia.ceph.com]: reset os type and version on server 2022-03-26T23:52:07.844 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2022-03-26T23:52:07.862 INFO:teuthology.lock.ops:Node 'ubuntu@smithi122.front.sepia.ceph.com' reimaging is complete 2022-03-26T23:52:07.863 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging 2022-03-26T23:52:07.863 INFO:teuthology.lock.ops:Updating [ubuntu@smithi174.front.sepia.ceph.com]: reset os type and version on server 2022-03-26T23:52:07.864 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2022-03-26T23:52:07.882 INFO:teuthology.lock.ops:Node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging is complete 2022-03-26T23:52:08.158 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of ubuntu 20.04 2022-03-26T23:52:08.162 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of ubuntu 20.04 2022-03-26T23:52:08.624 INFO:teuthology.orchestra.console:Power off smithi122 2022-03-26T23:52:08.677 INFO:teuthology.orchestra.console:Power off smithi174 2022-03-26T23:52:08.872 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2022-03-26T23:52:08.973 INFO:teuthology.orchestra.console:Power on smithi174 2022-03-26T23:52:16.992 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2022-03-26T23:52:17.093 INFO:teuthology.orchestra.console:Power on smithi122 2022-03-26T23:52:17.336 INFO:teuthology.orchestra.console:Power on for smithi174 completed 2022-03-26T23:52:17.437 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2022-03-26T23:52:25.464 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2022-03-26T23:52:25.565 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2022-03-26T23:57:18.265 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-26T23:57:18.721 INFO:teuthology.provision.fog.smithi174:Node is ready 2022-03-26T23:57:18.728 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2022-03-26T23:57:18.735 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2022-03-26T23:57:18.901 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2022-03-26T23:57:32.188 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-26T23:57:32.667 INFO:teuthology.provision.fog.smithi122:Node is ready 2022-03-26T23:57:32.674 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122.front.sepia.ceph.com 2022-03-26T23:57:32.681 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122 smithi122.front.sepia.ceph.com smithi122 2022-03-26T23:57:32.906 INFO:teuthology.provision.fog.smithi122:Deploy complete! 2022-03-26T23:57:33.080 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com 2022-03-26T23:57:33.081 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-26T23:57:33.098 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2022-03-26T23:57:33.099 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-26T23:57:33.127 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server 2022-03-26T23:57:33.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2022-03-26T23:57:33.351 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2022-03-26T23:57:33.358 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2022-03-26T23:57:33.358 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-26T23:57:33.358 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2022-03-26T23:57:33.359 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2022-03-26T23:57:33.359 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-26T23:57:33.359 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="20.04" 2022-03-26T23:57:33.359 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-26T23:57:33.360 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-26T23:57:33.360 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-26T23:57:33.361 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-26T23:57:33.361 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=focal 2022-03-26T23:57:33.361 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=focal 2022-03-26T23:57:33.362 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2022-03-26T23:57:33.379 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2022-03-26T23:57:33.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2022-03-26T23:57:33.554 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2022-03-26T23:57:33.561 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu" 2022-03-26T23:57:33.561 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-26T23:57:33.562 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu 2022-03-26T23:57:33.562 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian 2022-03-26T23:57:33.562 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-26T23:57:33.562 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="20.04" 2022-03-26T23:57:33.563 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-26T23:57:33.563 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-26T23:57:33.563 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-26T23:57:33.563 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-26T23:57:33.564 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=focal 2022-03-26T23:57:33.564 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=focal 2022-03-26T23:57:33.564 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2022-03-26T23:57:33.657 INFO:teuthology.dispatcher.supervisor:Running job 6757602 2022-03-26T23:57:33.657 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757602 --name vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/valgrind} -- /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757602/orig.config.yaml 2022-03-26T23:57:33.660 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757602 2022-03-26T23:57:33.661 INFO:teuthology.dispatcher.supervisor:Job PID: 22693 2022-03-26T23:57:33.661 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-27T06:28:02.415 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-27T06:28:02.593 INFO:teuthology.kill:Killing Pids: {22693} 2022-03-27T06:28:03.836 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-27T06:28:03.837 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-27T06:28:03.837 INFO:teuthology.misc:Compressing logs... 2022-03-27T06:28:03.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2022-03-27T06:28:04.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2022-03-27T06:28:04.542 INFO:teuthology.misc:Compressing logs... 2022-03-27T06:28:04.695 INFO:teuthology.misc:Compressing logs... 2022-03-27T06:28:12.308 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/ceph-osd.4.log: file size changed while zipping 2022-03-27T06:28:21.671 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /var/log/ceph/ceph-mds.a.log: file size changed while zipping 2022-03-27T06:28:22.032 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/ceph-osd.7.log: file size changed while zipping 2022-03-27T06:28:28.838 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /var/log/ceph/ceph-osd.3.log: file size changed while zipping 2022-03-27T06:28:30.955 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/ceph-osd.6.log: file size changed while zipping 2022-03-27T06:28:36.177 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/ceph-osd.5.log: file size changed while zipping 2022-03-27T06:29:37.571 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /var/log/ceph/ceph-osd.0.log: file size changed while zipping 2022-03-27T06:29:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /var/log/ceph/ceph-osd.1.log: file size changed while zipping 2022-03-27T06:29:47.520 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /var/log/ceph/ceph-osd.2.log: file size changed while zipping 2022-03-27T06:30:25.134 INFO:teuthology.kill:No teuthology processes running 2022-03-27T06:30:26.294 INFO:teuthology.kill:Nuking machines: ['smithi122', 'smithi174'] 2022-03-27T06:30:27.310 INFO:teuthology.kill:2022-03-27 06:30:27,309.309 INFO:teuthology.nuke:targets: 2022-03-27T06:30:27.311 INFO:teuthology.kill: smithi122.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyrOEkcryqgXI246/W7R2oHPndCOvvYP1lGJX7szIEBpqQS3lmn4PlcIotqJhx8B5jISJBptXWgUbR7piJXxKuaPOxYr+rgE/iG5kC3+hYfpTpMmEXq5brjc4N6xMyw5olxMXTEZqSuPCJwN/oekz6OeTiV4aSXyfa1w4O/ri161WhBY2h2JCqNvVSRjgcmYJeym0mxIgzQk+FNpetc0q+KrOzebnPkCWe0y60mBq3PX3ZTc98aUz1/oxBUWTVpQCs/x8TiJKCgqgRFJG49SMSAWipBZpB6pYhvV6udp0tU8HU4UrKikCJQCTtm3DrH3aA4CluZJWnrCJXYZukdyrmZ6XN5vxABhBN/m5c9ufsFekknr+Szq7ylboZ+288HBFPvA//soHRO8dn1hYJ6UgI2maZnb0ftVHr0US66ZQDSdN+Y4UjVE5rNXy4jl0+BoqoTmZ46LRlFGs4rIYIib4HXfwHFDrlasyfI4MYoo4DE+TIjIEgIC4FaqUT/jfgQ3M= 2022-03-27T06:30:27.311 INFO:teuthology.kill: smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIfP8UGzmRlgd/A42/5+Kk7Qa1qQrRgUjRd7fNaWMZx7g80OYtxFcgi6rStG69gk5OnQC4HAaQAlxiP7c2Wu2mKH2xx9f2Wy3Wr3VRFtqc/RVLjN2QuE3Aamnn5BpOK4OrzG/RbRR5tvFmR6+zX6LV0F5sayOp1Xz0JynTtnaCFykdfVp07oURdhfm70Too/SCpdAF3eBZ7BEWnu475PB/+jGIoHvrj9fWbyQ38VDw3uF0vUR9RwULQtHT8id545kpb4Ru9NGvn0w6eAboeBO1aUvhmNm2tios+EiRc7T9AqC6Fl3NAEfcCUHdcFEu1ToP9JFJvINAPfaYQPQi0I3ZPpLR+x1GMknphWpXphQGCtKLQTWKZAjYf0wpbUdYRdgKT90Y/FGO6TB/Z01j2xmTVRBjT5MO9ySokA94QJEtvCAxmOYfXXTDkedjsUSSgF5xEe+Bmwo5eePl1+3qPRKxcSIPhxGiJQ5dlbzyQmjHAWm9WR9gxvoN6hO/zWCpWM8= 2022-03-27T06:30:27.336 INFO:teuthology.kill:2022-03-27 06:30:27,336.336 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-27T06:30:27.339 INFO:teuthology.kill:2022-03-27 06:30:27,339.339 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-27T06:30:27.415 INFO:teuthology.kill:2022-03-27 06:30:27,415.415 INFO:teuthology.orchestra.console:Power off smithi174 2022-03-27T06:30:27.428 INFO:teuthology.kill:2022-03-27 06:30:27,428.428 INFO:teuthology.orchestra.console:Power off smithi122 2022-03-27T06:30:35.787 INFO:teuthology.kill:2022-03-27 06:30:35,786.786 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2022-03-27T06:30:35.944 INFO:teuthology.kill:2022-03-27 06:30:35,943.943 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com 2022-03-27T06:30:39.905 INFO:teuthology.kill:2022-03-27 06:30:39,905.905 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2022-03-27T06:30:40.050 INFO:teuthology.kill:2022-03-27 06:30:40,050.050 INFO:teuthology.lock.ops:unlocked: smithi122.front.sepia.ceph.com 2022-03-27T06:32:40.370 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-27T06:32:40.486 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi122 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-03-20_03:31:03-rados-pacific-distro-default-smithi/6748053 2022-03-27T06:32:40.486 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi174 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-03-20_03:31:03-rados-pacific-distro-default-smithi/6748051