2022-03-26T16:25:19.293 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-26T16:25:19.460 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi111.front.sepia.ceph.com' reimaging 2022-03-26T16:25:19.461 INFO:teuthology.lock.ops:Updating [ubuntu@smithi111.front.sepia.ceph.com]: reset os type and version on server 2022-03-26T16:25:19.462 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2022-03-26T16:25:19.487 INFO:teuthology.lock.ops:Node 'ubuntu@smithi111.front.sepia.ceph.com' reimaging is complete 2022-03-26T16:25:19.487 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi141.front.sepia.ceph.com' reimaging 2022-03-26T16:25:19.487 INFO:teuthology.lock.ops:Updating [ubuntu@smithi141.front.sepia.ceph.com]: reset os type and version on server 2022-03-26T16:25:19.488 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2022-03-26T16:25:19.509 INFO:teuthology.lock.ops:Node 'ubuntu@smithi141.front.sepia.ceph.com' reimaging is complete 2022-03-26T16:25:19.698 INFO:teuthology.provision.fog.smithi111:Scheduling deploy of centos 8.stream 2022-03-26T16:25:19.713 INFO:teuthology.provision.fog.smithi141:Scheduling deploy of centos 8.stream 2022-03-26T16:25:20.062 INFO:teuthology.orchestra.console:Power off smithi141 2022-03-26T16:25:20.107 INFO:teuthology.orchestra.console:Power off smithi111 2022-03-26T16:25:32.564 INFO:teuthology.orchestra.console:Power off for smithi141 completed 2022-03-26T16:25:32.584 INFO:teuthology.orchestra.console:Power off for smithi111 completed 2022-03-26T16:25:32.666 INFO:teuthology.orchestra.console:Power on smithi141 2022-03-26T16:25:32.685 INFO:teuthology.orchestra.console:Power on smithi111 2022-03-26T16:25:36.938 INFO:teuthology.orchestra.console:Power on for smithi141 completed 2022-03-26T16:25:36.946 INFO:teuthology.orchestra.console:Power on for smithi111 completed 2022-03-26T16:25:37.040 INFO:teuthology.provision.fog.smithi141:Waiting for deploy to finish 2022-03-26T16:25:37.048 INFO:teuthology.provision.fog.smithi111:Waiting for deploy to finish 2022-03-26T16:29:49.408 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-26T16:30:14.683 INFO:teuthology.provision.fog.smithi141:Node is ready 2022-03-26T16:30:14.729 INFO:teuthology.orchestra.run.smithi141.stdout:smithi141.front.sepia.ceph.com 2022-03-26T16:30:14.830 INFO:teuthology.orchestra.run.smithi141.stdout:172.21.15.141 smithi141.front.sepia.ceph.com smithi141 2022-03-26T16:30:15.222 INFO:teuthology.provision.fog.smithi141:Deploy complete! 2022-03-26T16:30:22.665 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-26T16:30:48.374 INFO:teuthology.provision.fog.smithi111:Node is ready 2022-03-26T16:30:48.397 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111.front.sepia.ceph.com 2022-03-26T16:30:48.458 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111 smithi111.front.sepia.ceph.com smithi111 2022-03-26T16:30:48.839 INFO:teuthology.provision.fog.smithi111:Deploy complete! 2022-03-26T16:30:49.031 INFO:teuthology.lock.ops:Checking smithi111.front.sepia.ceph.com 2022-03-26T16:30:49.032 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-26T16:30:49.050 INFO:teuthology.lock.ops:Checking smithi141.front.sepia.ceph.com 2022-03-26T16:30:49.051 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-26T16:30:49.068 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: set os type and version on server 2022-03-26T16:30:49.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2022-03-26T16:30:49.275 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2022-03-26T16:30:49.333 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2022-03-26T16:30:49.333 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="8" 2022-03-26T16:30:49.333 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2022-03-26T16:30:49.333 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2022-03-26T16:30:49.334 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="8" 2022-03-26T16:30:49.334 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el8" 2022-03-26T16:30:49.334 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-26T16:30:49.334 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2022-03-26T16:30:49.335 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-26T16:30:49.335 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2022-03-26T16:30:49.335 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-26T16:30:49.335 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-26T16:30:49.336 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-26T16:30:49.337 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2022-03-26T16:30:49.355 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: set os type and version on server 2022-03-26T16:30:49.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2022-03-26T16:30:49.598 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64 2022-03-26T16:30:49.654 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="CentOS Stream" 2022-03-26T16:30:49.655 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8" 2022-03-26T16:30:49.655 INFO:teuthology.orchestra.run.smithi141.stdout:ID="centos" 2022-03-26T16:30:49.655 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="rhel fedora" 2022-03-26T16:30:49.656 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8" 2022-03-26T16:30:49.656 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8" 2022-03-26T16:30:49.656 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-26T16:30:49.657 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31" 2022-03-26T16:30:49.657 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-26T16:30:49.657 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://centos.org/" 2022-03-26T16:30:49.658 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-26T16:30:49.658 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-26T16:30:49.658 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-26T16:30:49.659 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2022-03-26T16:30:49.807 INFO:teuthology.dispatcher.supervisor:Running job 6757479 2022-03-26T16:30:49.808 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757479 --name vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/valgrind} -- /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757479/orig.config.yaml 2022-03-26T16:30:49.811 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757479 2022-03-26T16:30:49.812 INFO:teuthology.dispatcher.supervisor:Job PID: 2211 2022-03-26T16:30:49.813 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-26T23:01:17.737 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-26T23:01:17.902 INFO:teuthology.kill:Killing Pids: {2211} 2022-03-26T23:01:19.172 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-26T23:01:19.173 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-26T23:01:19.174 INFO:teuthology.misc:Compressing logs... 2022-03-26T23:01:19.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2022-03-26T23:01:19.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2022-03-26T23:01:19.819 INFO:teuthology.misc:Compressing logs... 2022-03-26T23:01:20.056 INFO:teuthology.misc:Compressing logs... 2022-03-26T23:01:33.560 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/ceph-osd.5.log: file size changed while zipping 2022-03-26T23:01:36.717 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/ceph-osd.7.log: file size changed while zipping 2022-03-26T23:01:39.027 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/ceph-mds.e.log: file size changed while zipping 2022-03-26T23:01:48.011 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/ceph-osd.0.log: file size changed while zipping 2022-03-26T23:01:51.569 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/ceph-osd.2.log: file size changed while zipping 2022-03-26T23:02:08.940 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/ceph-osd.3.log: file size changed while zipping 2022-03-26T23:02:12.473 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/ceph-osd.1.log: file size changed while zipping 2022-03-26T23:02:16.998 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/ceph-osd.6.log: file size changed while zipping 2022-03-26T23:02:43.235 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/ceph-osd.4.log: file size changed while zipping 2022-03-26T23:03:31.618 INFO:teuthology.kill:No teuthology processes running 2022-03-26T23:03:32.768 INFO:teuthology.kill:Nuking machines: ['smithi111', 'smithi141'] 2022-03-26T23:03:33.720 INFO:teuthology.kill:2022-03-26 23:03:33,720.720 INFO:teuthology.nuke:targets: 2022-03-26T23:03:33.721 INFO:teuthology.kill: smithi111.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN8Hnj4NwbFFR7JWsNIne8hDvq11aa42IiiOjIaeTd/d5bP3/ZePr3nhK5ilbtsOevYihc79gQXHkneLt2A1WslsGxpX7pre8Ev7fui5ylOt1cWfGIbubwZ3ztbNBXbJMhMzvvMdBXKuSzFI+PxAEfPx3Fh6RHXdiuDU0WNL/eoW87Whs2uflqQmt1TVUa+O5Kg3FBQvXA2yMiJOC5EEC0wlnzYlgYtwRqO/UsWgjywfYFBPdc6CXVTYNXyoatmTMNNXSIIMfdyznwL29Aen/qVJc5OQLRw/aXSAHVZeXPyvGvaWzLPmVAfBBfgfxjozLUCAprlfT02CYpyMZijRm4LC37V0qahxJ4KZANJplCexhOW/lDrkXc6Nu6JciWaMB8xO6fuGjdBuqJi9gmSFJFSrmpJQgz6j0dYNC+OyNNYs0yI8XKTfXgUgYt0GSCGoSBz/lcWB+lEBVz7A+pKg8l8N7ohEtp3TbYxE5dKA9ToyFR4nFV4AMno2BECpC9O/c= 2022-03-26T23:03:33.721 INFO:teuthology.kill: smithi141.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvhMqapZr69O6noi24MVRnBAhQTA2cerqLc/fPZRIetC7Tct6BQh9zsGIpawcAccKBmlLpAm7ihSr8gBs2d9PUnDzRQvA3wbRhkjq4zI5/2iE7j612n9LTJWeowK5tYEeZ2+gYfFPO13wW97P/YBJdPz3gtc+R0Ooo8wGEzVoNTgmsFDpCi6KX4/tNY/cFuVTzWPBLIoQBvdQaZ9K96H4OuzLVbbDWe1DuAiWiqw8qCXU/6iPn0Yya1peP2Jmu2VGxRK1qIjrVn+KXxT8kG/+9DVh1V6b3PRAhVLthVDC3V/BPFk1h9TnUAmreMk1T1RpHiZZkcuKmmYktOO+oLWKyesWWNo1Ic0qza/l5Yt1w/MugKu43nV3+u21Vz5zSfgdfShEwqIf/IlROmVNZeoDi+DdG+rLaeCTK2DRLm7A0pk9pyWaLndMYP5dea7hN/6vcypeaQvxzMLoi9VRiYm+/bFX6Z+Pq1rOXbpw7rDsBTwKTghzCgxbunPN/YCI6Sz8= 2022-03-26T23:03:33.746 INFO:teuthology.kill:2022-03-26 23:03:33,746.746 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-26T23:03:33.749 INFO:teuthology.kill:2022-03-26 23:03:33,749.749 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-26T23:03:33.833 INFO:teuthology.kill:2022-03-26 23:03:33,833.833 INFO:teuthology.orchestra.console:Power off smithi141 2022-03-26T23:03:33.846 INFO:teuthology.kill:2022-03-26 23:03:33,846.846 INFO:teuthology.orchestra.console:Power off smithi111 2022-03-26T23:03:42.206 INFO:teuthology.kill:2022-03-26 23:03:42,206.206 INFO:teuthology.orchestra.console:Power off for smithi141 completed 2022-03-26T23:03:42.356 INFO:teuthology.kill:2022-03-26 23:03:42,355.355 INFO:teuthology.lock.ops:unlocked: smithi141.front.sepia.ceph.com 2022-03-26T23:03:46.324 INFO:teuthology.kill:2022-03-26 23:03:46,323.323 INFO:teuthology.orchestra.console:Power off for smithi111 completed 2022-03-26T23:03:46.470 INFO:teuthology.kill:2022-03-26 23:03:46,470.470 INFO:teuthology.lock.ops:unlocked: smithi111.front.sepia.ceph.com 2022-03-26T23:05:46.779 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-26T23:05:46.912 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi111 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/6762757 2022-03-26T23:05:46.913 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi141 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/6762757