2022-04-03T22:20:23.179 INFO:root:teuthology version: 1.1.0-d877b023
2022-04-03T22:20:23.306 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi016.front.sepia.ceph.com' reimaging
2022-04-03T22:20:23.307 INFO:teuthology.lock.ops:Updating [ubuntu@smithi016.front.sepia.ceph.com]: reset os type and version on server
2022-04-03T22:20:23.308 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server
2022-04-03T22:20:23.325 INFO:teuthology.lock.ops:Node 'ubuntu@smithi016.front.sepia.ceph.com' reimaging is complete
2022-04-03T22:20:23.326 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging
2022-04-03T22:20:23.326 INFO:teuthology.lock.ops:Updating [ubuntu@smithi120.front.sepia.ceph.com]: reset os type and version on server
2022-04-03T22:20:23.326 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2022-04-03T22:20:23.345 INFO:teuthology.lock.ops:Node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging is complete
2022-04-03T22:20:23.532 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of rhel 8.4
2022-04-03T22:20:23.563 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.4
2022-04-03T22:20:23.853 INFO:teuthology.orchestra.console:Power off smithi016
2022-04-03T22:20:23.895 INFO:teuthology.orchestra.console:Power off smithi120
2022-04-03T22:20:24.005 INFO:teuthology.orchestra.console:Power off for smithi016 completed
2022-04-03T22:20:24.036 INFO:teuthology.orchestra.console:Power off for smithi120 completed
2022-04-03T22:20:24.106 INFO:teuthology.orchestra.console:Power on smithi016
2022-04-03T22:20:24.138 INFO:teuthology.orchestra.console:Power on smithi120
2022-04-03T22:20:28.358 INFO:teuthology.orchestra.console:Power on for smithi016 completed
2022-04-03T22:20:28.384 INFO:teuthology.orchestra.console:Power on for smithi120 completed
2022-04-03T22:20:28.460 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish
2022-04-03T22:20:28.485 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish
2022-04-03T22:22:44.503 ERROR:teuthology.orchestra.connection:Error authenticating with smithi016.front.sepia.ceph.com: Authentication failed.
2022-04-03T22:24:16.247 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-03T22:24:31.249 INFO:teuthology.provision.fog.smithi016:Node is ready
2022-04-03T22:24:31.269 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com
2022-04-03T22:24:31.327 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016
2022-04-03T22:24:31.673 INFO:teuthology.provision.fog.smithi016:Deploy complete!
2022-04-03T22:24:47.074 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-03T22:25:07.693 INFO:teuthology.provision.fog.smithi120:Node is ready
2022-04-03T22:25:07.712 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com
2022-04-03T22:25:07.768 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120
2022-04-03T22:25:08.100 INFO:teuthology.provision.fog.smithi120:Deploy complete!
2022-04-03T22:25:08.248 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com
2022-04-03T22:25:08.249 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-03T22:25:08.269 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com
2022-04-03T22:25:08.269 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-03T22:25:08.285 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server
2022-04-03T22:25:08.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com'
2022-04-03T22:25:08.530 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64
2022-04-03T22:25:08.586 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux"
2022-04-03T22:25:08.586 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.4 (Ootpa)"
2022-04-03T22:25:08.586 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel"
2022-04-03T22:25:08.586 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora"
2022-04-03T22:25:08.587 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.4"
2022-04-03T22:25:08.587 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8"
2022-04-03T22:25:08.587 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-04-03T22:25:08.587 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31"
2022-04-03T22:25:08.588 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-04-03T22:25:08.588 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/"
2022-04-03T22:25:08.588 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-04-03T22:25:08.588 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-03T22:25:08.589 INFO:teuthology.orchestra.run.smithi016.stdout:
2022-04-03T22:25:08.589 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-03T22:25:08.589 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-04-03T22:25:08.589 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-04-03T22:25:08.589 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-04-03T22:25:08.591 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server
2022-04-03T22:25:08.614 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server
2022-04-03T22:25:08.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2022-04-03T22:25:08.837 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64
2022-04-03T22:25:08.893 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux"
2022-04-03T22:25:08.893 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.4 (Ootpa)"
2022-04-03T22:25:08.893 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel"
2022-04-03T22:25:08.893 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora"
2022-04-03T22:25:08.894 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.4"
2022-04-03T22:25:08.894 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8"
2022-04-03T22:25:08.894 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-04-03T22:25:08.894 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31"
2022-04-03T22:25:08.895 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-04-03T22:25:08.895 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/"
2022-04-03T22:25:08.895 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-04-03T22:25:08.895 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-03T22:25:08.895 INFO:teuthology.orchestra.run.smithi120.stdout:
2022-04-03T22:25:08.896 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-03T22:25:08.896 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-04-03T22:25:08.896 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-04-03T22:25:08.896 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-04-03T22:25:08.897 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2022-04-03T22:25:08.999 INFO:teuthology.dispatcher.supervisor:Running job 6775702
2022-04-03T22:25:08.999 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-04-03_15:22:48-fs-wip-vshankar-testing-20220403-170344-testing-default-smithi/6775702 --name vshankar-2022-04-03_15:22:48-fs-wip-vshankar-testing-20220403-170344-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} -- /home/teuthworker/archive/vshankar-2022-04-03_15:22:48-fs-wip-vshankar-testing-20220403-170344-testing-default-smithi/6775702/orig.config.yaml
2022-04-03T22:25:09.003 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-04-03_15:22:48-fs-wip-vshankar-testing-20220403-170344-testing-default-smithi/6775702
2022-04-03T22:25:09.003 INFO:teuthology.dispatcher.supervisor:Job PID: 28332
2022-04-03T22:25:09.004 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-04T04:55:19.631 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-04-04T04:55:19.787 INFO:teuthology.kill:Killing Pids: {28332}
2022-04-04T04:55:20.861 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-04-04T04:55:20.862 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-04-04T04:55:20.863 INFO:teuthology.misc:Compressing logs...
2022-04-04T04:55:20.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com'
2022-04-04T04:55:21.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2022-04-04T04:55:21.577 INFO:teuthology.misc:Compressing logs...
2022-04-04T04:57:24.181 INFO:teuthology.misc:Compressing logs...
2022-04-04T05:00:23.165 INFO:teuthology.kill:No teuthology processes running
2022-04-04T05:00:24.358 INFO:teuthology.kill:Nuking machines: ['smithi016', 'smithi120']
2022-04-04T05:00:25.327 INFO:teuthology.kill:2022-04-04 05:00:25,327.327 INFO:teuthology.nuke:targets:
2022-04-04T05:00:25.328 INFO:teuthology.kill:  smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGpy5Eju86lGquNXNEW17wbQh5IOchPuArZAXAVAGPwhSfjodbAQ5/lsbnM7MPS09GEMEnDv2B4Ufn0w0tmGdadzDhNeGC209B8pQ6rVx6K2rvbk/Ta5acHz7cC3uu/fDrrhohbQeEoCmiPhtoUViR1VK3rMfckjqmwUgaio8BDy/Q5aTJfGC/NKNoXH9cJjfNTUsUlnhu427THpkK+7ba/aE8Bnmt8C5CdVaYMGDyaS39P8XWuwhekrJoMRIbuzEKX4a4zhzbHvaKsbWS3KdxBtcNRzNReR1fRhy+NN3/rRn+Fus3nEHRl8E04rG7YuZy6cFfH2uL3wiSDm/svhYpNTsxmG0OGOIWprqw6Z5ZjflEikcHn8XaCViP73A1HuZIlk0gB+fKl2xS5gfJ7fA0kxCGkQjv1jNJX+Ssdkq6BYrFp7raR8sbQFyemMbo2GxWCHU+XUxU0An2QRIXZkdYPFOqev8aPHf5037p/2/c46l1lklMxndwbrZVidUKmNE=
2022-04-04T05:00:25.329 INFO:teuthology.kill:  smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0DWLBxwjhFILqv3YwfvBL3sM5b6cmbGv+/pfQeGMG3X+4bP42P7FOj/8ucPCUgwm9uWSQ9CP3b/Hjo2d7xJn+cUt7aXLmCQvUvX6TtKN2/7QbLj5aSXWHbhse0K1yLC+9BXTnbzNXOtwzo95K9dpdMJAIB9pj4RnHelCouekO+YxyWdfH4/qeaGWxenC0WQfWMjx/whh5Hhe8aWzzYecp6s83geXw/GWrvxb7eFP2zfc0dkKQNwiFMFtxkSrvNBWwGcqc2ICocDTeuirzBOmM5jBVgLw8oQ8v/XtGSfkzVubPXE4vLX/DoBKKqaMZZPehbfTh5Q/X+Y3BbILywo7ynJRn14HdNjaUV7xd4Y455Jo6SlTjQRSDfbt1MGbY0CIMpHLP1r1q1PBjexZIILUFTnhFtCtoZrtUMoHpjZoM6OrXroe1Kc51nGTumUzt/12NztBkOFyUp9MTvQFwMBZeTj4D5ms/alr521+ZnfFvgWnjs4P8iOeueuTCGU2U50=
2022-04-04T05:00:25.355 INFO:teuthology.kill:2022-04-04 05:00:25,355.355 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-04T05:00:25.359 INFO:teuthology.kill:2022-04-04 05:00:25,359.359 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-04T05:00:25.437 INFO:teuthology.kill:2022-04-04 05:00:25,437.437 INFO:teuthology.orchestra.console:Power off smithi120
2022-04-04T05:00:25.450 INFO:teuthology.kill:2022-04-04 05:00:25,450.450 INFO:teuthology.orchestra.console:Power off smithi016
2022-04-04T05:00:33.808 INFO:teuthology.kill:2022-04-04 05:00:33,807.807 INFO:teuthology.orchestra.console:Power off for smithi016 completed
2022-04-04T05:00:33.959 INFO:teuthology.kill:2022-04-04 05:00:33,958.958 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com
2022-04-04T05:00:37.913 INFO:teuthology.kill:2022-04-04 05:00:37,913.913 INFO:teuthology.orchestra.console:Power off for smithi120 completed
2022-04-04T05:00:38.060 INFO:teuthology.kill:2022-04-04 05:00:38,060.060 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com
2022-04-04T05:02:38.295 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-04T05:02:38.410 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi016 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-01_03:20:03-krbd-quincy-testing-default-smithi/6771526
2022-04-04T05:02:38.410 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi120 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-01_03:20:03-krbd-quincy-testing-default-smithi/6771525