2022-03-19T01:29:00.770 INFO:root:teuthology version: 1.1.0-aa078dbe
2022-03-19T01:29:00.890 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging
2022-03-19T01:29:00.891 INFO:teuthology.lock.ops:Updating [ubuntu@smithi121.front.sepia.ceph.com]: reset os type and version on server
2022-03-19T01:29:00.892 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2022-03-19T01:29:00.910 INFO:teuthology.lock.ops:Node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging is complete
2022-03-19T01:29:00.911 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging
2022-03-19T01:29:00.911 INFO:teuthology.lock.ops:Updating [ubuntu@smithi149.front.sepia.ceph.com]: reset os type and version on server
2022-03-19T01:29:00.912 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2022-03-19T01:29:00.929 INFO:teuthology.lock.ops:Node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging is complete
2022-03-19T01:29:01.129 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of rhel 8.4
2022-03-19T01:29:01.150 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of rhel 8.4
2022-03-19T01:29:01.488 INFO:teuthology.orchestra.console:Power off smithi121
2022-03-19T01:29:01.528 INFO:teuthology.orchestra.console:Power off smithi149
2022-03-19T01:29:13.973 INFO:teuthology.orchestra.console:Power off for smithi121 completed
2022-03-19T01:29:13.998 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2022-03-19T01:29:14.075 INFO:teuthology.orchestra.console:Power on smithi121
2022-03-19T01:29:14.103 INFO:teuthology.orchestra.console:Power on smithi149
2022-03-19T01:29:18.331 INFO:teuthology.orchestra.console:Power on for smithi121 completed
2022-03-19T01:29:18.432 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish
2022-03-19T01:29:22.464 INFO:teuthology.orchestra.console:Power on for smithi149 completed
2022-03-19T01:29:22.565 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish
2022-03-19T01:33:37.088 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-19T01:33:57.541 INFO:teuthology.provision.fog.smithi121:Node is ready
2022-03-19T01:33:57.560 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com
2022-03-19T01:33:57.617 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121
2022-03-19T01:33:57.975 INFO:teuthology.provision.fog.smithi121:Deploy complete!
2022-03-19T01:34:00.567 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-19T01:34:00.791 INFO:teuthology.provision.fog.smithi149:Node is ready
2022-03-19T01:34:00.805 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com
2022-03-19T01:34:00.861 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149
2022-03-19T01:34:01.190 INFO:teuthology.provision.fog.smithi149:Deploy complete!
2022-03-19T01:34:01.373 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com
2022-03-19T01:34:01.374 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-19T01:34:01.394 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com
2022-03-19T01:34:01.395 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-19T01:34:01.414 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server
2022-03-19T01:34:01.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com'
2022-03-19T01:34:01.650 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64
2022-03-19T01:34:01.705 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux"
2022-03-19T01:34:01.705 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.4 (Ootpa)"
2022-03-19T01:34:01.705 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel"
2022-03-19T01:34:01.706 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora"
2022-03-19T01:34:01.706 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.4"
2022-03-19T01:34:01.706 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8"
2022-03-19T01:34:01.706 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-19T01:34:01.707 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31"
2022-03-19T01:34:01.707 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-19T01:34:01.707 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/"
2022-03-19T01:34:01.707 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-19T01:34:01.708 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-19T01:34:01.708 INFO:teuthology.orchestra.run.smithi121.stdout:
2022-03-19T01:34:01.708 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-19T01:34:01.708 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-19T01:34:01.708 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-19T01:34:01.709 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-19T01:34:01.710 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2022-03-19T01:34:01.730 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server
2022-03-19T01:34:01.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2022-03-19T01:34:01.949 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64
2022-03-19T01:34:02.003 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Red Hat Enterprise Linux"
2022-03-19T01:34:02.004 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8.4 (Ootpa)"
2022-03-19T01:34:02.004 INFO:teuthology.orchestra.run.smithi149.stdout:ID="rhel"
2022-03-19T01:34:02.004 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="fedora"
2022-03-19T01:34:02.004 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8.4"
2022-03-19T01:34:02.005 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8"
2022-03-19T01:34:02.005 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-19T01:34:02.005 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2022-03-19T01:34:02.005 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-19T01:34:02.005 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.redhat.com/"
2022-03-19T01:34:02.006 INFO:teuthology.orchestra.run.smithi149.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-19T01:34:02.006 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-19T01:34:02.006 INFO:teuthology.orchestra.run.smithi149.stdout:
2022-03-19T01:34:02.006 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-19T01:34:02.007 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-19T01:34:02.007 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-19T01:34:02.007 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-19T01:34:02.008 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2022-03-19T01:34:02.109 INFO:teuthology.dispatcher.supervisor:Running job 6744723
2022-03-19T01:34:02.110 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_aa078dbed6d14c1ad98beb5a02b956bbe0a741c8/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744723 --name vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} -- /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744723/orig.config.yaml
2022-03-19T01:34:02.113 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744723
2022-03-19T01:34:02.114 INFO:teuthology.dispatcher.supervisor:Job PID: 23334
2022-03-19T01:34:02.114 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-19T08:04:29.852 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-19T08:04:30.008 INFO:teuthology.kill:Killing Pids: {23334}
2022-03-19T08:04:31.251 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-19T08:04:31.252 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-19T08:04:31.252 INFO:teuthology.misc:Compressing logs...
2022-03-19T08:04:31.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com'
2022-03-19T08:04:31.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2022-03-19T08:04:32.121 INFO:teuthology.misc:Compressing logs...
2022-03-19T08:04:32.327 INFO:teuthology.misc:Compressing logs...
2022-03-19T08:06:47.665 INFO:teuthology.kill:No teuthology processes running
2022-03-19T08:06:48.805 INFO:teuthology.kill:Nuking machines: ['smithi121', 'smithi149']
2022-03-19T08:06:49.779 INFO:teuthology.kill:2022-03-19 08:06:49,778.778 INFO:teuthology.nuke:targets:
2022-03-19T08:06:49.780 INFO:teuthology.kill:  smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiVLZjBQX/HyuFFeTLIHzs3VTW8ZvtH2iQ/JWghl0QJQyToAtBt/HbvT3tAOr/YIzwbdGFvK3L9imM/CsBJN1WvDbeq+HHcnRtG6DBAPTAzL/gloCTv9iOd4J6v4twxWXxukzoD8zSR5Oc0NZgeKdJMAO2eyXrAzfu69+QgVwy06w/mvFUa5BFCeCYydKkFumdeFhChoJjzipDllXjXxFTHVWFPL/G+7fZTHXunEqnwfMhjZ3nBbqZ/74pvZYYi6QbbeMwc58LOXHzS7++vQEJurRzPJADUdYKiIw0opXcTslIBKH0UYs8EOtURg2ISONBTGEgShJih1lzUdrQWXmbAZRg172p2Imms7vGm55fei+vX7bTRR2jebFXkln2DZeL2fE+UhnDHjCENWwj2pH1vFfN5cOMFFdUMlPN6z+eOAOZsuBqOYVbKIkOVCCTamGXdqgNiIv4h3hcnIWvaM7dlXUOCiRyQa9XbAJHRzODQWojol6C0jqBtc0DYnCS7TE=
2022-03-19T08:06:49.780 INFO:teuthology.kill:  smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiJzo97TvWsttRtRjLf20avIJr9KNwoSKUtHagnMk5NJeZ4snz6HYsEg7TPsljRXe6zJKqcQN9MQsJrDYUFxJK6/QmeUF4FpbGt1rPOupPb4FXuzungQ0ournAN0/Peu9zANgmhLucv+iCHWEk8M/kTq5fjhwJB5udbT4Oyi19e8Yj0qeYpMFQxSj6mnqQphcGAN5auGlVgV4v2aSmImZ86IsmfW2eORvItXm4E6dhy+HSjRanvkGNVHyTvtOHjl9t+eCVLxcrnDsqcuV9OASSc8C26UOjyd0TXIl0cQmpivp6VOmpZ5cVtFLohAhXzDps0NGGPLQeTXqxuXajEzGbrBcgQLnHupa21de1Ul6KqV66YWrM57c5Nm2PhZ7nb/+dY6qmk4LVaSBAFpMqNh8mZeAAocDU+4glmQl9YWnEZtuxTr9e+aXQD5CRtjAonO9r7I14VA+YFQUmBBAJdW3tjHrKrX8yglT0argjG83pAb1v18LGL92llJNeZHbykas=
2022-03-19T08:06:49.806 INFO:teuthology.kill:2022-03-19 08:06:49,806.806 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-19T08:06:49.808 INFO:teuthology.kill:2022-03-19 08:06:49,808.808 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-19T08:06:49.885 INFO:teuthology.kill:2022-03-19 08:06:49,885.885 INFO:teuthology.orchestra.console:Power off smithi149
2022-03-19T08:06:49.899 INFO:teuthology.kill:2022-03-19 08:06:49,899.899 INFO:teuthology.orchestra.console:Power off smithi121
2022-03-19T08:07:02.370 INFO:teuthology.kill:2022-03-19 08:07:02,370.370 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2022-03-19T08:07:02.375 INFO:teuthology.kill:2022-03-19 08:07:02,375.375 INFO:teuthology.orchestra.console:Power off for smithi121 completed
2022-03-19T08:07:02.523 INFO:teuthology.kill:2022-03-19 08:07:02,522.522 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com
2022-03-19T08:07:02.526 INFO:teuthology.kill:2022-03-19 08:07:02,526.526 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com
2022-03-19T08:09:02.860 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-19T08:09:02.970 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi121 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744873
2022-03-19T08:09:02.971 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi149 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744873