2022-03-14T09:44:57.895 INFO:root:teuthology version: 1.1.0-fbfc7b38
2022-03-14T09:44:58.028 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi067.front.sepia.ceph.com' reimaging
2022-03-14T09:44:58.029 INFO:teuthology.lock.ops:Updating [ubuntu@smithi067.front.sepia.ceph.com]: reset os type and version on server
2022-03-14T09:44:58.029 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2022-03-14T09:44:58.048 INFO:teuthology.lock.ops:Node 'ubuntu@smithi067.front.sepia.ceph.com' reimaging is complete
2022-03-14T09:44:58.049 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging
2022-03-14T09:44:58.049 INFO:teuthology.lock.ops:Updating [ubuntu@smithi082.front.sepia.ceph.com]: reset os type and version on server
2022-03-14T09:44:58.049 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server
2022-03-14T09:44:58.071 INFO:teuthology.lock.ops:Node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging is complete
2022-03-14T09:44:58.292 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of rhel 8.4
2022-03-14T09:44:58.294 INFO:teuthology.provision.fog.smithi067:Scheduling deploy of rhel 8.4
2022-03-14T09:44:58.648 INFO:teuthology.orchestra.console:Power off smithi082
2022-03-14T09:44:58.662 INFO:teuthology.orchestra.console:Power off smithi067
2022-03-14T09:44:58.791 INFO:teuthology.orchestra.console:Power off for smithi082 completed
2022-03-14T09:44:58.804 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2022-03-14T09:44:58.893 INFO:teuthology.orchestra.console:Power on smithi082
2022-03-14T09:44:58.907 INFO:teuthology.orchestra.console:Power on smithi067
2022-03-14T09:45:07.264 INFO:teuthology.orchestra.console:Power on for smithi082 completed
2022-03-14T09:45:07.268 INFO:teuthology.orchestra.console:Power on for smithi067 completed
2022-03-14T09:45:07.366 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish
2022-03-14T09:45:07.370 INFO:teuthology.provision.fog.smithi067:Waiting for deploy to finish
2022-03-14T09:49:28.639 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-14T09:49:39.039 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-14T09:49:39.286 INFO:teuthology.provision.fog.smithi082:Node is ready
2022-03-14T09:49:39.301 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082.front.sepia.ceph.com
2022-03-14T09:49:39.356 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82 smithi082.front.sepia.ceph.com smithi082
2022-03-14T09:49:39.682 INFO:teuthology.provision.fog.smithi082:Deploy complete!
2022-03-14T09:49:48.870 INFO:teuthology.provision.fog.smithi067:Node is ready
2022-03-14T09:49:48.890 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067.front.sepia.ceph.com
2022-03-14T09:49:48.948 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67 smithi067.front.sepia.ceph.com smithi067
2022-03-14T09:49:49.295 INFO:teuthology.provision.fog.smithi067:Deploy complete!
2022-03-14T09:49:49.455 INFO:teuthology.lock.ops:Checking smithi067.front.sepia.ceph.com
2022-03-14T09:49:49.456 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-14T09:49:49.476 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com
2022-03-14T09:49:49.477 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-14T09:49:49.495 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: set os type and version on server
2022-03-14T09:49:49.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com'
2022-03-14T09:49:49.726 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64
2022-03-14T09:49:49.781 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="Red Hat Enterprise Linux"
2022-03-14T09:49:49.782 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8.4 (Ootpa)"
2022-03-14T09:49:49.782 INFO:teuthology.orchestra.run.smithi067.stdout:ID="rhel"
2022-03-14T09:49:49.782 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="fedora"
2022-03-14T09:49:49.783 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8.4"
2022-03-14T09:49:49.783 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8"
2022-03-14T09:49:49.783 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-14T09:49:49.783 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31"
2022-03-14T09:49:49.784 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-14T09:49:49.784 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://www.redhat.com/"
2022-03-14T09:49:49.784 INFO:teuthology.orchestra.run.smithi067.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-14T09:49:49.784 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-14T09:49:49.785 INFO:teuthology.orchestra.run.smithi067.stdout:
2022-03-14T09:49:49.785 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-14T09:49:49.785 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-14T09:49:49.785 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-14T09:49:49.785 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-14T09:49:49.787 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2022-03-14T09:49:49.806 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server
2022-03-14T09:49:49.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com'
2022-03-14T09:49:50.042 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64
2022-03-14T09:49:50.099 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux"
2022-03-14T09:49:50.099 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.4 (Ootpa)"
2022-03-14T09:49:50.099 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel"
2022-03-14T09:49:50.100 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora"
2022-03-14T09:49:50.100 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.4"
2022-03-14T09:49:50.100 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8"
2022-03-14T09:49:50.100 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-14T09:49:50.101 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31"
2022-03-14T09:49:50.101 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-14T09:49:50.101 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/"
2022-03-14T09:49:50.101 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-14T09:49:50.101 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-14T09:49:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:
2022-03-14T09:49:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-14T09:49:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-14T09:49:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-14T09:49:50.103 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-14T09:49:50.103 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server
2022-03-14T09:49:50.210 INFO:teuthology.dispatcher.supervisor:Running job 6735721
2022-03-14T09:49:50.211 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735721 --name vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}} -- /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735721/orig.config.yaml
2022-03-14T09:49:50.214 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735721
2022-03-14T09:49:50.215 INFO:teuthology.dispatcher.supervisor:Job PID: 15041
2022-03-14T09:49:50.215 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-14T16:20:19.035 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-14T16:20:19.212 INFO:teuthology.kill:Killing Pids: {15041}
2022-03-14T16:20:20.354 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-14T16:20:20.355 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-14T16:20:20.355 INFO:teuthology.misc:Compressing logs...
2022-03-14T16:20:20.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com'
2022-03-14T16:20:20.595 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com'
2022-03-14T16:20:21.003 INFO:teuthology.misc:Compressing logs...
2022-03-14T16:20:21.225 INFO:teuthology.misc:Compressing logs...
2022-03-14T16:29:18.742 INFO:teuthology.kill:No teuthology processes running
2022-03-14T16:29:19.878 INFO:teuthology.kill:Nuking machines: ['smithi067', 'smithi082']
2022-03-14T16:29:20.872 INFO:teuthology.kill:2022-03-14 16:29:20,871.871 INFO:teuthology.nuke:targets:
2022-03-14T16:29:20.872 INFO:teuthology.kill:  smithi067.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9SXlfu7b5QrcfYGdiWurm3X6pjicLxcvmkSiPMhKf37W5lZNSYgmXEhNrfS2o10KJFlVJIWnWbGS3JeBm4JIij4EXj2ZPvVTl/YMsOLbHirIW8r9kLGEqFnUO7HMXw4JLNYg82jkWikmJy58P/JNuUahcx+A5IjCkkO7R6amGDtVIUkWF6LBEkn1jWvpB/cLXtkJ9OxFjIgFBnxeey1KKaGw0xesAEA1gACnVEt0GM9rP00BkFx6QO/k402lCl+Hczh6RaBSCgmf8hprU0CVskgdUbGM9QBVeqj8P61RW0TgqZ19Bx/EV0yYilEDd/H3fd4uGVyhCvVn8+JJqL+nD1Nl0FH6ZaLUUksjTWxy2PMylzm36/VncOHoCzMuul3OZ7yPVn49ilHTbFb/hehL+fCaBTqw6aWIc4qOK61mxUVmr1pSUakvegNpmxzDHxVnPGKTbrAut4mFU1SShisfzvuPgTS9mrIgx5tAcPZ47Uwzdef9V7bxIdw4HI8xWDHU=
2022-03-14T16:29:20.873 INFO:teuthology.kill:  smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvZN4sOowvB7TYZrXy4Z8gYtiX+LFEVEPOuZ6FaebrakKkUEVUM3wkncQHIaJd9fFih1plEFv39WMwO185vTd7cXOk4H1Mv3hxIuwaMJjZKRu+w8ZQBwnQyq/q3ZHeJc5hXp6EWj8Lvt2inoIAhw8dfUnXW3MgdaV1vajGXdZoNDL1gvt8licHcLW8y2fn13EHv6SOSCk2zT9MR8L0Q462uNhRrNfmgBml1dxOstPwYjYPMyK+JDSMxiGQIHIEks4x/XIi7yUuM7KWmC2KcBza03OuMpeXzcAuOaLpFpP2O5XaQDMnSgJctnjE6h1YOG3503B/xSF2vLjR/VCPIV5brz4ZEBFDaTmjles03uAsvKhsZ5MuvqzVXqrWM/S0iU6ImKQmHaY9Xslda167PBhq48eq0hjCnhvML7dKlIIDxD3Ay+41ATIROeMptTTd6TJyOgiqwDbvvg3EvBhrILhdzCNfAQMhU39pZYhMeE9yZAB5H3gBAwh6ra0Xg9wRm7k=
2022-03-14T16:29:20.899 INFO:teuthology.kill:2022-03-14 16:29:20,899.899 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-14T16:29:20.902 INFO:teuthology.kill:2022-03-14 16:29:20,902.902 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-14T16:29:20.980 INFO:teuthology.kill:2022-03-14 16:29:20,980.980 INFO:teuthology.orchestra.console:Power off smithi082
2022-03-14T16:29:20.994 INFO:teuthology.kill:2022-03-14 16:29:20,993.993 INFO:teuthology.orchestra.console:Power off smithi067
2022-03-14T16:29:29.352 INFO:teuthology.kill:2022-03-14 16:29:29,352.352 INFO:teuthology.orchestra.console:Power off for smithi082 completed
2022-03-14T16:29:29.498 INFO:teuthology.kill:2022-03-14 16:29:29,497.497 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com
2022-03-14T16:29:33.470 INFO:teuthology.kill:2022-03-14 16:29:33,470.470 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2022-03-14T16:29:33.617 INFO:teuthology.kill:2022-03-14 16:29:33,617.617 INFO:teuthology.lock.ops:unlocked: smithi067.front.sepia.ceph.com
2022-03-14T16:31:33.923 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-14T16:31:34.035 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi067 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735931
2022-03-14T16:31:34.035 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi082 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735931