2022-03-21T10:08:22.627 INFO:root:teuthology version: 1.1.0-a4e18662
2022-03-21T10:08:22.747 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging
2022-03-21T10:08:22.748 INFO:teuthology.lock.ops:Updating [ubuntu@smithi077.front.sepia.ceph.com]: reset os type and version on server
2022-03-21T10:08:22.748 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server
2022-03-21T10:08:22.765 INFO:teuthology.lock.ops:Node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging is complete
2022-03-21T10:08:22.766 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi204.front.sepia.ceph.com' reimaging
2022-03-21T10:08:22.766 INFO:teuthology.lock.ops:Updating [ubuntu@smithi204.front.sepia.ceph.com]: reset os type and version on server
2022-03-21T10:08:22.766 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server
2022-03-21T10:08:22.784 INFO:teuthology.lock.ops:Node 'ubuntu@smithi204.front.sepia.ceph.com' reimaging is complete
2022-03-21T10:08:22.972 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of rhel 8.4
2022-03-21T10:08:23.010 INFO:teuthology.provision.fog.smithi204:Scheduling deploy of rhel 8.4
2022-03-21T10:08:23.288 INFO:teuthology.orchestra.console:Power off smithi077
2022-03-21T10:08:23.327 INFO:teuthology.orchestra.console:Power off smithi204
2022-03-21T10:08:23.464 INFO:teuthology.orchestra.console:Power off for smithi204 completed
2022-03-21T10:08:23.565 INFO:teuthology.orchestra.console:Power on smithi204
2022-03-21T10:08:31.660 INFO:teuthology.orchestra.console:Power off for smithi077 completed
2022-03-21T10:08:31.762 INFO:teuthology.orchestra.console:Power on smithi077
2022-03-21T10:08:31.924 INFO:teuthology.orchestra.console:Power on for smithi204 completed
2022-03-21T10:08:32.025 INFO:teuthology.provision.fog.smithi204:Waiting for deploy to finish
2022-03-21T10:08:40.123 INFO:teuthology.orchestra.console:Power on for smithi077 completed
2022-03-21T10:08:40.224 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish
2022-03-21T10:11:02.955 ERROR:teuthology.orchestra.connection:Error authenticating with smithi204.front.sepia.ceph.com: Authentication failed.
2022-03-21T10:13:11.777 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-21T10:13:12.004 INFO:teuthology.provision.fog.smithi077:Node is ready
2022-03-21T10:13:12.018 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com
2022-03-21T10:13:12.074 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077
2022-03-21T10:13:12.390 INFO:teuthology.provision.fog.smithi077:Deploy complete!
2022-03-21T10:13:16.230 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-21T10:13:16.461 INFO:teuthology.provision.fog.smithi204:Node is ready
2022-03-21T10:13:16.476 INFO:teuthology.orchestra.run.smithi204.stdout:smithi204.front.sepia.ceph.com
2022-03-21T10:13:16.531 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204
2022-03-21T10:13:16.846 INFO:teuthology.provision.fog.smithi204:Deploy complete!
2022-03-21T10:13:17.001 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com
2022-03-21T10:13:17.002 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-21T10:13:17.021 INFO:teuthology.lock.ops:Checking smithi204.front.sepia.ceph.com
2022-03-21T10:13:17.022 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-21T10:13:17.040 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server
2022-03-21T10:13:17.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com'
2022-03-21T10:13:17.268 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64
2022-03-21T10:13:17.323 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="Red Hat Enterprise Linux"
2022-03-21T10:13:17.324 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8.4 (Ootpa)"
2022-03-21T10:13:17.324 INFO:teuthology.orchestra.run.smithi077.stdout:ID="rhel"
2022-03-21T10:13:17.324 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="fedora"
2022-03-21T10:13:17.324 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8.4"
2022-03-21T10:13:17.325 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8"
2022-03-21T10:13:17.325 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-21T10:13:17.325 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31"
2022-03-21T10:13:17.325 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-21T10:13:17.326 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://www.redhat.com/"
2022-03-21T10:13:17.326 INFO:teuthology.orchestra.run.smithi077.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-21T10:13:17.326 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-21T10:13:17.326 INFO:teuthology.orchestra.run.smithi077.stdout:
2022-03-21T10:13:17.326 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-21T10:13:17.327 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-21T10:13:17.327 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-21T10:13:17.327 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-21T10:13:17.328 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server
2022-03-21T10:13:17.347 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: set os type and version on server
2022-03-21T10:13:17.348 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com'
2022-03-21T10:13:17.566 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64
2022-03-21T10:13:17.620 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Red Hat Enterprise Linux"
2022-03-21T10:13:17.621 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8.4 (Ootpa)"
2022-03-21T10:13:17.621 INFO:teuthology.orchestra.run.smithi204.stdout:ID="rhel"
2022-03-21T10:13:17.621 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="fedora"
2022-03-21T10:13:17.621 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8.4"
2022-03-21T10:13:17.621 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8"
2022-03-21T10:13:17.622 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-21T10:13:17.622 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31"
2022-03-21T10:13:17.622 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-21T10:13:17.622 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.redhat.com/"
2022-03-21T10:13:17.623 INFO:teuthology.orchestra.run.smithi204.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-21T10:13:17.623 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-21T10:13:17.623 INFO:teuthology.orchestra.run.smithi204.stdout:
2022-03-21T10:13:17.623 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-21T10:13:17.624 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-21T10:13:17.624 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-21T10:13:17.624 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-21T10:13:17.624 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server
2022-03-21T10:13:17.722 INFO:teuthology.dispatcher.supervisor:Running job 6749871
2022-03-21T10:13:17.722 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6749871 --name vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} -- /home/teuthworker/archive/vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6749871/orig.config.yaml
2022-03-21T10:13:17.725 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-21_06:07:09-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6749871
2022-03-21T10:13:17.727 INFO:teuthology.dispatcher.supervisor:Job PID: 18734
2022-03-21T10:13:17.727 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-21T16:43:28.265 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-21T16:43:28.428 INFO:teuthology.kill:Killing Pids: {18734}
2022-03-21T16:43:29.619 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-21T16:43:29.620 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-21T16:43:29.621 INFO:teuthology.misc:Compressing logs...
2022-03-21T16:43:29.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com'
2022-03-21T16:43:29.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com'
2022-03-21T16:43:30.223 INFO:teuthology.misc:Compressing logs...
2022-03-21T16:43:30.376 INFO:teuthology.misc:Compressing logs...
2022-03-21T16:45:22.436 INFO:teuthology.kill:No teuthology processes running
2022-03-21T16:45:23.630 INFO:teuthology.kill:Nuking machines: ['smithi077', 'smithi204']
2022-03-21T16:45:24.614 INFO:teuthology.kill:2022-03-21 16:45:24,614.614 INFO:teuthology.nuke:targets:
2022-03-21T16:45:24.615 INFO:teuthology.kill:  smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC80DRaHoUF4Dr33GGuKvZIsIeuRTsuMDe44HklvUXsne0PBuYdB7BKD4S2l9GKAeOf7gQgXcL8vXcLcnbTChaYg41ok69jvpqOlz8oSMuz7PwPzY1S2qkse4O5BmY5HNkNdlaxNVcfFBbzgLZi2ztmO68i84hZvPoK9kn/HZwb3WiyFzmdioRNE4cllJGD6yenc510PQASGnB0sp2UXWLEVv8cYSppcJ0ZQGxVLK5Aj5UqIanCg/bG2vFfsazGGqcij8o57yqQf9dy2SdeafdPq/bDwbtf5rc10PvPA0Ny7Cym14ACKLRlFFsH5DKURkZpuflXLPO1TTjKF+GmIPncNoz5mgEqkE32Q0ft3ywVuEA4YnNN5DDRJzAqnwzy21CMe0822zkAHj8h1mafLpsbYloRDqSvg8I3SQMcMOhH+vH4368RQ+6a7TAGrTEmGdEv2b4eH9JAwlgNVNZoy4bu3en2f7HR0jWUgMcJ17PFu8aEDdmTltPwKBvltFEqoas=
2022-03-21T16:45:24.616 INFO:teuthology.kill:  smithi204.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsqTArd+OCiAyeI+9Jpsg3c2DEJslHlGY5HZJ3zUYuTS1uZfhRoQZ3SEeNgrSi3Jaf+MQWaergdHt3UnaQuZcA25ehey8h5C/gKG1rbS8Tl6DtaeYFC5Wjcv3kZ+mSx+XPZUFOlJUrNrXrUxb0dJhKjeTYuYxDqy+4qChMRle9AKKntUg8XscJx3FagU19TUuMleJFPXAY3HBm3iy3r4LD/GNpBw9JUQyL3Wu+VMWEVCBlRoh0sxvc40Kp6VL3JceVnVaxOmv0zQFWLNwqYBbhhzDbN85bjLFf79nYYZBepK/Ag+euW/ov0gsshtp2rTu5AKyifu2AOCwygZSnVXIad6/jBK+qzofcw8LU6dtJ+myAtXKLJB+ix5pqPMsM1hHoahNhYg9rTsRqyEDPIuHAIt5dSUmHV1RZQYsdNyqGTY26C6rYo3KjkOBZ1Nkdxqs5a0ThKLPAaG7/FXyLaITUnCAeadD6bH+SPHkUVNSKCDecooejSe5zxhfxR38k/o8=
2022-03-21T16:45:24.641 INFO:teuthology.kill:2022-03-21 16:45:24,641.641 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-21T16:45:24.644 INFO:teuthology.kill:2022-03-21 16:45:24,644.644 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-21T16:45:24.720 INFO:teuthology.kill:2022-03-21 16:45:24,720.720 INFO:teuthology.orchestra.console:Power off smithi077
2022-03-21T16:45:24.733 INFO:teuthology.kill:2022-03-21 16:45:24,733.733 INFO:teuthology.orchestra.console:Power off smithi204
2022-03-21T16:45:33.093 INFO:teuthology.kill:2022-03-21 16:45:33,092.092 INFO:teuthology.orchestra.console:Power off for smithi204 completed
2022-03-21T16:45:33.236 INFO:teuthology.kill:2022-03-21 16:45:33,235.235 INFO:teuthology.lock.ops:unlocked: smithi204.front.sepia.ceph.com
2022-03-21T16:45:37.199 INFO:teuthology.kill:2022-03-21 16:45:37,199.199 INFO:teuthology.orchestra.console:Power off for smithi077 completed
2022-03-21T16:45:37.345 INFO:teuthology.kill:2022-03-21 16:45:37,345.345 INFO:teuthology.lock.ops:unlocked: smithi077.front.sepia.ceph.com
2022-03-21T16:47:37.577 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-21T16:47:37.715 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi077 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-21_13:12:20-fs-master-testing-default-smithi/6750240
2022-03-21T16:47:37.716 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi204 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-21_13:12:20-fs-master-testing-default-smithi/6750239