2022-03-21T06:55:44.528 INFO:root:teuthology version: 1.1.0-a4e18662
2022-03-21T06:55:44.646 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi160.front.sepia.ceph.com' reimaging
2022-03-21T06:55:44.647 INFO:teuthology.lock.ops:Updating [ubuntu@smithi160.front.sepia.ceph.com]: reset os type and version on server
2022-03-21T06:55:44.648 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server
2022-03-21T06:55:44.668 INFO:teuthology.lock.ops:Node 'ubuntu@smithi160.front.sepia.ceph.com' reimaging is complete
2022-03-21T06:55:44.669 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi171.front.sepia.ceph.com' reimaging
2022-03-21T06:55:44.669 INFO:teuthology.lock.ops:Updating [ubuntu@smithi171.front.sepia.ceph.com]: reset os type and version on server
2022-03-21T06:55:44.670 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server
2022-03-21T06:55:44.690 INFO:teuthology.lock.ops:Node 'ubuntu@smithi171.front.sepia.ceph.com' reimaging is complete
2022-03-21T06:55:44.882 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of rhel 8.4
2022-03-21T06:55:44.889 INFO:teuthology.provision.fog.smithi171:Scheduling deploy of rhel 8.4
2022-03-21T06:55:45.221 INFO:teuthology.orchestra.console:Power off smithi160
2022-03-21T06:55:45.236 INFO:teuthology.orchestra.console:Power off smithi171
2022-03-21T06:55:53.594 INFO:teuthology.orchestra.console:Power off for smithi160 completed
2022-03-21T06:55:53.695 INFO:teuthology.orchestra.console:Power on smithi160
2022-03-21T06:55:57.712 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2022-03-21T06:55:57.813 INFO:teuthology.orchestra.console:Power on smithi171
2022-03-21T06:55:57.945 INFO:teuthology.orchestra.console:Power on for smithi160 completed
2022-03-21T06:55:58.046 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish
2022-03-21T06:56:02.070 INFO:teuthology.orchestra.console:Power on for smithi171 completed
2022-03-21T06:56:02.171 INFO:teuthology.provision.fog.smithi171:Waiting for deploy to finish
2022-03-21T07:00:19.224 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-21T07:00:39.682 INFO:teuthology.provision.fog.smithi160:Node is ready
2022-03-21T07:00:39.701 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com
2022-03-21T07:00:39.757 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160
2022-03-21T07:00:40.113 INFO:teuthology.provision.fog.smithi160:Deploy complete!
2022-03-21T07:00:55.087 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-21T07:00:55.316 INFO:teuthology.provision.fog.smithi171:Node is ready
2022-03-21T07:00:55.334 INFO:teuthology.orchestra.run.smithi171.stdout:smithi171.front.sepia.ceph.com
2022-03-21T07:00:55.389 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.15.171 smithi171.front.sepia.ceph.com smithi171
2022-03-21T07:00:55.711 INFO:teuthology.provision.fog.smithi171:Deploy complete!
2022-03-21T07:00:55.903 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com
2022-03-21T07:00:55.904 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-21T07:00:55.923 INFO:teuthology.lock.ops:Checking smithi171.front.sepia.ceph.com
2022-03-21T07:00:55.923 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-21T07:00:55.941 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server
2022-03-21T07:00:55.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com'
2022-03-21T07:00:56.175 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64
2022-03-21T07:00:56.230 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Red Hat Enterprise Linux"
2022-03-21T07:00:56.231 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8.4 (Ootpa)"
2022-03-21T07:00:56.231 INFO:teuthology.orchestra.run.smithi160.stdout:ID="rhel"
2022-03-21T07:00:56.231 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="fedora"
2022-03-21T07:00:56.231 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8.4"
2022-03-21T07:00:56.232 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8"
2022-03-21T07:00:56.232 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-21T07:00:56.232 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31"
2022-03-21T07:00:56.232 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-21T07:00:56.233 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.redhat.com/"
2022-03-21T07:00:56.233 INFO:teuthology.orchestra.run.smithi160.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-21T07:00:56.233 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-21T07:00:56.233 INFO:teuthology.orchestra.run.smithi160.stdout:
2022-03-21T07:00:56.234 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-21T07:00:56.234 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-21T07:00:56.234 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-21T07:00:56.234 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-21T07:00:56.235 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server
2022-03-21T07:00:56.254 INFO:teuthology.lock.ops:Updating [smithi171.front.sepia.ceph.com]: set os type and version on server
2022-03-21T07:00:56.254 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com'
2022-03-21T07:00:56.478 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64
2022-03-21T07:00:56.533 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="Red Hat Enterprise Linux"
2022-03-21T07:00:56.533 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="8.4 (Ootpa)"
2022-03-21T07:00:56.533 INFO:teuthology.orchestra.run.smithi171.stdout:ID="rhel"
2022-03-21T07:00:56.534 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="fedora"
2022-03-21T07:00:56.534 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="8.4"
2022-03-21T07:00:56.534 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el8"
2022-03-21T07:00:56.534 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-21T07:00:56.535 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31"
2022-03-21T07:00:56.535 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-21T07:00:56.535 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://www.redhat.com/"
2022-03-21T07:00:56.535 INFO:teuthology.orchestra.run.smithi171.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-21T07:00:56.536 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-21T07:00:56.536 INFO:teuthology.orchestra.run.smithi171.stdout:
2022-03-21T07:00:56.536 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-21T07:00:56.536 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-21T07:00:56.536 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-21T07:00:56.537 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-21T07:00:56.537 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server
2022-03-21T07:00:56.639 INFO:teuthology.dispatcher.supervisor:Running job 6749777
2022-03-21T07:00:56.640 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6749777 --name vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} -- /home/teuthworker/archive/vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6749777/orig.config.yaml
2022-03-21T07:00:56.645 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6749777
2022-03-21T07:00:56.645 INFO:teuthology.dispatcher.supervisor:Job PID: 3491
2022-03-21T07:00:56.646 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-21T13:31:24.320 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-21T13:31:24.517 INFO:teuthology.kill:Killing Pids: {3491}
2022-03-21T13:31:25.604 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-21T13:31:25.605 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-21T13:31:25.605 INFO:teuthology.misc:Compressing logs...
2022-03-21T13:31:25.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com'
2022-03-21T13:31:27.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com'
2022-03-21T13:31:28.106 INFO:teuthology.misc:Compressing logs...
2022-03-21T13:31:28.324 INFO:teuthology.misc:Compressing logs...
2022-03-21T13:40:52.988 INFO:teuthology.kill:No teuthology processes running
2022-03-21T13:40:54.276 INFO:teuthology.kill:Nuking machines: ['smithi160', 'smithi171']
2022-03-21T13:40:55.248 INFO:teuthology.kill:2022-03-21 13:40:55,248.248 INFO:teuthology.nuke:targets:
2022-03-21T13:40:55.249 INFO:teuthology.kill:  smithi160.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvibS62FHAu9PvJbja6YPexP01hw5h2ED3qRsumOXSvPYeC+E1of0VuZg3tfeTExNsIMxoS4HumkpYE9yk/Nz3XdcooG/aDquW38UbPS9GznvtSzYpWF7T1dV8Hj3+ame+vzaom8UgJGLMTWMdm/oGgWrQaC3B2V2T9+qRNEfjhpc4GWkwzm0cfPP2ISrgjxBjCYYutBjdKcZg2okugrEfmwwVjtcZZstHZyw5cF6IXaPm4ad/2GxapJ4EwCxtYOE4WTtoAcJVf8Abgdm/7QZvUnCUi9dC2es1LpCzW4PSMjQq3E4DEAdO+ZKKn/IYeKqo2NYkdM4LZyjZoehaoYk2x82sVNBNbcJQr92cVwvRUPHrMtTdA7SxHcjuNF1skPOfJMxBUCa93wcP1jSyjZgpnE0r2LfhzpKfyihJ3S18g8ffQQTRDVcKpfbHiqrtxnDwMgvUlftxVbM93LbeDyUPbo8NIY2BcPPgTQRnohnwNY2azmLdjl6hXdvXZ4zaa7c=
2022-03-21T13:40:55.249 INFO:teuthology.kill:  smithi171.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCIMMt64WTj5NdOiAuAmM3QGF7ACX1x4/50Q8xO9jUjPCutoI1BY9z78BqLkiHlNCC+gINJDZNB1OJK4DZO8cw2uAzlbmm6lE/+Z+/RA8wohkRCa0wghOKqPD7i3Oude8T/OKtbTpWPawvZ6a7wkbHWsfNFt+OtZYzz8ELvJ8PCvKmNWN1u/8ZOFP0bJD2tN0iJnMlAIw2qBO4qLQao5KVt7O3KrjqJn9+GmwS25yugfNx6pK9SqNFDhj8g7NsbUJ7xb4lIpnWEQZ540V+Qn3pvazYo9z2loJZj1MXbbY4xHvCzqVoRHXcwfUbbXMHEMPbJnIhILWJdtNo7ouYSdDlUgPv/cNdDlzayDbC+x/AGZ6dmtOKe2H4Fa+N0AnBujT50GEOoCThjQCvDXDw06oqsCKjD4+izPi+yaVbjHnRrwyymCVJ4UHacuzkqWg4xQw8d+gmWXa11IwKGSAWLunV2iXBSgl+E06w5DMGAIAPZ4UTz0rMqF2wc7ezuEOJv0c=
2022-03-21T13:40:55.274 INFO:teuthology.kill:2022-03-21 13:40:55,273.273 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-21T13:40:55.276 INFO:teuthology.kill:2022-03-21 13:40:55,276.276 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-21T13:40:55.346 INFO:teuthology.kill:2022-03-21 13:40:55,346.346 INFO:teuthology.orchestra.console:Power off smithi160
2022-03-21T13:40:55.359 INFO:teuthology.kill:2022-03-21 13:40:55,359.359 INFO:teuthology.orchestra.console:Power off smithi171
2022-03-21T13:41:07.823 INFO:teuthology.kill:2022-03-21 13:41:07,823.823 INFO:teuthology.orchestra.console:Power off for smithi160 completed
2022-03-21T13:41:07.830 INFO:teuthology.kill:2022-03-21 13:41:07,829.829 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2022-03-21T13:41:07.968 INFO:teuthology.kill:2022-03-21 13:41:07,967.967 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com
2022-03-21T13:41:07.975 INFO:teuthology.kill:2022-03-21 13:41:07,974.974 INFO:teuthology.lock.ops:unlocked: smithi171.front.sepia.ceph.com
2022-03-21T13:43:08.255 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-21T13:43:08.361 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi160 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6749908
2022-03-21T13:43:08.362 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi171 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6749908