2022-03-19T03:41:05.059 INFO:root:teuthology version: 1.1.0-aa078dbe
2022-03-19T03:41:05.179 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi101.front.sepia.ceph.com' reimaging
2022-03-19T03:41:05.180 INFO:teuthology.lock.ops:Updating [ubuntu@smithi101.front.sepia.ceph.com]: reset os type and version on server
2022-03-19T03:41:05.180 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2022-03-19T03:41:05.199 INFO:teuthology.lock.ops:Node 'ubuntu@smithi101.front.sepia.ceph.com' reimaging is complete
2022-03-19T03:41:05.200 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi143.front.sepia.ceph.com' reimaging
2022-03-19T03:41:05.200 INFO:teuthology.lock.ops:Updating [ubuntu@smithi143.front.sepia.ceph.com]: reset os type and version on server
2022-03-19T03:41:05.201 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server
2022-03-19T03:41:05.219 INFO:teuthology.lock.ops:Node 'ubuntu@smithi143.front.sepia.ceph.com' reimaging is complete
2022-03-19T03:41:05.412 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of rhel 8.4
2022-03-19T03:41:05.426 INFO:teuthology.provision.fog.smithi143:Scheduling deploy of rhel 8.4
2022-03-19T03:41:05.694 INFO:teuthology.orchestra.console:Power off smithi101
2022-03-19T03:41:05.729 INFO:teuthology.orchestra.console:Power off smithi143
2022-03-19T03:41:14.062 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2022-03-19T03:41:14.088 INFO:teuthology.orchestra.console:Power off for smithi143 completed
2022-03-19T03:41:14.163 INFO:teuthology.orchestra.console:Power on smithi101
2022-03-19T03:41:14.189 INFO:teuthology.orchestra.console:Power on smithi143
2022-03-19T03:41:22.527 INFO:teuthology.orchestra.console:Power on for smithi101 completed
2022-03-19T03:41:22.606 INFO:teuthology.orchestra.console:Power on for smithi143 completed
2022-03-19T03:41:22.629 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish
2022-03-19T03:41:22.707 INFO:teuthology.provision.fog.smithi143:Waiting for deploy to finish
2022-03-19T03:46:00.533 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-19T03:46:00.551 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-19T03:46:00.765 INFO:teuthology.provision.fog.smithi143:Node is ready
2022-03-19T03:46:00.783 INFO:teuthology.provision.fog.smithi101:Node is ready
2022-03-19T03:46:00.799 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com
2022-03-19T03:46:00.820 INFO:teuthology.orchestra.run.smithi143.stdout:smithi143.front.sepia.ceph.com
2022-03-19T03:46:00.854 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101
2022-03-19T03:46:00.875 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143 smithi143.front.sepia.ceph.com smithi143
2022-03-19T03:46:01.186 INFO:teuthology.provision.fog.smithi101:Deploy complete!
2022-03-19T03:46:01.200 INFO:teuthology.provision.fog.smithi143:Deploy complete!
2022-03-19T03:46:01.405 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com
2022-03-19T03:46:01.406 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-19T03:46:01.425 INFO:teuthology.lock.ops:Checking smithi143.front.sepia.ceph.com
2022-03-19T03:46:01.425 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-19T03:46:01.442 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server
2022-03-19T03:46:01.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2022-03-19T03:46:01.665 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64
2022-03-19T03:46:01.720 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Red Hat Enterprise Linux"
2022-03-19T03:46:01.720 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8.4 (Ootpa)"
2022-03-19T03:46:01.720 INFO:teuthology.orchestra.run.smithi101.stdout:ID="rhel"
2022-03-19T03:46:01.720 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="fedora"
2022-03-19T03:46:01.721 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8.4"
2022-03-19T03:46:01.721 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8"
2022-03-19T03:46:01.721 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-19T03:46:01.721 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31"
2022-03-19T03:46:01.722 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-19T03:46:01.722 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.redhat.com/"
2022-03-19T03:46:01.722 INFO:teuthology.orchestra.run.smithi101.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-19T03:46:01.722 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-19T03:46:01.723 INFO:teuthology.orchestra.run.smithi101.stdout:
2022-03-19T03:46:01.723 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-19T03:46:01.723 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-19T03:46:01.723 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-19T03:46:01.723 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-19T03:46:01.729 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2022-03-19T03:46:01.749 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: set os type and version on server
2022-03-19T03:46:01.749 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com'
2022-03-19T03:46:01.969 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64
2022-03-19T03:46:02.023 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Red Hat Enterprise Linux"
2022-03-19T03:46:02.024 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="8.4 (Ootpa)"
2022-03-19T03:46:02.024 INFO:teuthology.orchestra.run.smithi143.stdout:ID="rhel"
2022-03-19T03:46:02.024 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="fedora"
2022-03-19T03:46:02.024 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="8.4"
2022-03-19T03:46:02.025 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el8"
2022-03-19T03:46:02.025 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-19T03:46:02.025 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31"
2022-03-19T03:46:02.025 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-19T03:46:02.025 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.redhat.com/"
2022-03-19T03:46:02.026 INFO:teuthology.orchestra.run.smithi143.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-19T03:46:02.026 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-19T03:46:02.026 INFO:teuthology.orchestra.run.smithi143.stdout:
2022-03-19T03:46:02.026 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-19T03:46:02.027 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-19T03:46:02.027 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-19T03:46:02.027 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-19T03:46:02.027 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server
2022-03-19T03:46:02.126 INFO:teuthology.dispatcher.supervisor:Running job 6744803
2022-03-19T03:46:02.127 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_aa078dbed6d14c1ad98beb5a02b956bbe0a741c8/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744803 --name vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} -- /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744803/orig.config.yaml
2022-03-19T03:46:02.130 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744803
2022-03-19T03:46:02.131 INFO:teuthology.dispatcher.supervisor:Job PID: 15076
2022-03-19T03:46:02.132 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-19T10:16:29.243 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-19T10:16:29.444 INFO:teuthology.kill:Killing Pids: {15076}
2022-03-19T10:16:30.604 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-19T10:16:30.605 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-19T10:16:30.606 INFO:teuthology.misc:Compressing logs...
2022-03-19T10:16:30.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2022-03-19T10:16:31.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com'
2022-03-19T10:16:31.558 INFO:teuthology.misc:Compressing logs...
2022-03-19T10:16:31.784 INFO:teuthology.misc:Compressing logs...
2022-03-19T10:28:29.758 INFO:teuthology.kill:No teuthology processes running
2022-03-19T10:28:30.868 INFO:teuthology.kill:Nuking machines: ['smithi101', 'smithi143']
2022-03-19T10:28:32.010 INFO:teuthology.kill:2022-03-19 10:28:32,010.010 INFO:teuthology.nuke:targets:
2022-03-19T10:28:32.011 INFO:teuthology.kill:  smithi101.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC1BnuYlxRO1Ad2MYjsDaMs/XwbbELuYkDvhXX89uLWxptbpKQHT089HMc4R9vbaQw7o9igeBgTxNNKsxfV4Q7FLBCr75p5VjXA3I/5T0b8n2KTobv9OcZKMuTdgyDTqDeC3qNTHI6EEsh02QsqdF6XQva7sAgOFsje2susUXInlt7QRRhIx/L9br3W6mmrTuYWNhcjCEPBHr7MnKfsF9AnCYlzCfwQyWXf9luLJ0D0d2jcS6opSSwWy7aebn2+nOKYc0av58wT+wHtQZfIkGXF0gLlIY4ePSgjZm7cpdzrNKmiuW0uBAIJDjY8VS81ZpHWpd5AHKw7gNCvc0I1T7DSzE84rsy7RYGd+eIsFyZe/rKBh7mjlAVYhqY3LALyMOngATp4wu6ZOp/vnLfzgg3ckob9wRky4ptFs+16nH5P0YP6pQDDbj2r7juEyWQy2okC74cSFvjllOONUj7K+qMXqUlPgEZsdPoJFLP3MW16tqKQzM0m8cbGWCGRXObYrM=
2022-03-19T10:28:32.012 INFO:teuthology.kill:  smithi143.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdKbsjwIb4HjPUa0Px4RW0ElSOlKpE3lfPRP1pTL1wuCMnvw8k+NbjXW5H0Qq/Lbri71Tvas1fsdha/1Wa3rPUMYCpTwrvG0/ij5fYQTaApEQ10MrHEuioo3bzi9lVfSGJwT0UG0uvWgIUtyLkNCv89xl4sZ4ciSpGlFXu1JT3tC7u/k94BwacIX7ut/HnDnFBLVzaRb0OBA9MOqsG2Sj8Kxu3+uz52IwnpmdDRf9c4CvSA+XEsUnsgpLOu1LE0LGjvL973HoaerIAlCJLKXU83V0KWFLwWIDg2tnWHoY3kTXkkFt3LhAeQztCIeEWKUa/Nxn4r8IZvSN3Nj60GtDLuaAfiogk/pVS5/mHT0nS4RpxZcwqr95cChx4GIgGuBezfq0wTglF6tJ8tjgkpx4VlHu9jfPTnpzvzGrPDKfY10QpheoA87fdVAR6PsaBVsezTIPFG0FoUCxoYxo00ZMKli8fU2WM5F3GQLeYRStp9wltF44MVfFsf4puTpxz8J8=
2022-03-19T10:28:32.040 INFO:teuthology.kill:2022-03-19 10:28:32,040.040 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-19T10:28:32.044 INFO:teuthology.kill:2022-03-19 10:28:32,043.043 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-19T10:28:32.126 INFO:teuthology.kill:2022-03-19 10:28:32,125.125 INFO:teuthology.orchestra.console:Power off smithi101
2022-03-19T10:28:32.140 INFO:teuthology.kill:2022-03-19 10:28:32,139.139 INFO:teuthology.orchestra.console:Power off smithi143
2022-03-19T10:28:40.609 INFO:teuthology.kill:2022-03-19 10:28:40,608.608 INFO:teuthology.orchestra.console:Power off for smithi143 completed
2022-03-19T10:28:40.754 INFO:teuthology.kill:2022-03-19 10:28:40,753.753 INFO:teuthology.lock.ops:unlocked: smithi143.front.sepia.ceph.com
2022-03-19T10:28:44.668 INFO:teuthology.kill:2022-03-19 10:28:44,667.667 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2022-03-19T10:28:44.827 INFO:teuthology.kill:2022-03-19 10:28:44,827.827 INFO:teuthology.lock.ops:unlocked: smithi101.front.sepia.ceph.com
2022-03-19T10:30:45.117 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-19T10:30:45.224 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi101 but it was locked by another job: /home/teuthworker/archive/gsalomon-2022-03-19_10:12:44-rgw:verify-wip-test-arrow-submodule-distro-basic-smithi/6746453
2022-03-19T10:30:45.225 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi143 but it was locked by another job: /home/teuthworker/archive/gsalomon-2022-03-19_10:12:44-rgw:verify-wip-test-arrow-submodule-distro-basic-smithi/6746453