2022-05-02T09:28:48.718 INFO:root:teuthology version: 1.1.1.dev331+gfd5c3166
2022-05-02T09:28:48.861 INFO:teuthology.lock.ops:Start node 'smithi109.front.sepia.ceph.com' reimaging
2022-05-02T09:28:48.862 INFO:teuthology.lock.ops:Updating [smithi109.front.sepia.ceph.com]: reset os type and version on server
2022-05-02T09:28:48.862 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server
2022-05-02T09:28:48.885 INFO:teuthology.lock.ops:Node 'smithi109.front.sepia.ceph.com' reimaging is complete
2022-05-02T09:28:48.885 INFO:teuthology.lock.ops:Start node 'smithi124.front.sepia.ceph.com' reimaging
2022-05-02T09:28:48.886 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: reset os type and version on server
2022-05-02T09:28:48.886 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server
2022-05-02T09:28:48.907 INFO:teuthology.lock.ops:Node 'smithi124.front.sepia.ceph.com' reimaging is complete
2022-05-02T09:28:49.114 INFO:teuthology.provision.fog.smithi109:Scheduling deploy of rhel 8.5
2022-05-02T09:28:49.143 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of rhel 8.5
2022-05-02T09:28:49.476 INFO:teuthology.orchestra.console:Power off smithi109
2022-05-02T09:28:49.518 INFO:teuthology.orchestra.console:Power off smithi124
2022-05-02T09:29:01.968 INFO:teuthology.orchestra.console:Power off for smithi109 completed
2022-05-02T09:29:01.995 INFO:teuthology.orchestra.console:Power off for smithi124 completed
2022-05-02T09:29:02.070 INFO:teuthology.orchestra.console:Power on smithi109
2022-05-02T09:29:02.098 INFO:teuthology.orchestra.console:Power on smithi124
2022-05-02T09:29:06.329 INFO:teuthology.orchestra.console:Power on for smithi109 completed
2022-05-02T09:29:06.350 INFO:teuthology.orchestra.console:Power on for smithi124 completed
2022-05-02T09:29:06.431 INFO:teuthology.provision.fog.smithi109:Waiting for deploy to finish
2022-05-02T09:29:06.452 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish
2022-05-02T09:33:19.063 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-05-02T09:33:44.548 INFO:teuthology.provision.fog.smithi124:Node is ready
2022-05-02T09:33:44.569 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com
2022-05-02T09:33:44.629 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124
2022-05-02T09:33:45.008 INFO:teuthology.provision.fog.smithi124:Deploy complete!
2022-05-02T09:33:53.484 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-05-02T09:33:53.714 INFO:teuthology.provision.fog.smithi109:Node is ready
2022-05-02T09:33:53.730 INFO:teuthology.orchestra.run.smithi109.stdout:smithi109.front.sepia.ceph.com
2022-05-02T09:33:53.786 INFO:teuthology.orchestra.run.smithi109.stdout:172.21.15.109 smithi109.front.sepia.ceph.com smithi109
2022-05-02T09:33:54.111 INFO:teuthology.provision.fog.smithi109:Deploy complete!
2022-05-02T09:33:54.259 INFO:teuthology.lock.ops:Checking smithi109.front.sepia.ceph.com
2022-05-02T09:33:54.260 INFO:teuthology.lock.ops:New key found. Updating...
2022-05-02T09:33:54.278 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com
2022-05-02T09:33:54.278 INFO:teuthology.lock.ops:New key found. Updating...
2022-05-02T09:33:54.295 INFO:teuthology.lock.ops:Updating [smithi109.front.sepia.ceph.com]: set os type and version on server
2022-05-02T09:33:54.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com'
2022-05-02T09:33:54.487 INFO:teuthology.orchestra.run.smithi109.stdout:x86_64
2022-05-02T09:33:54.543 INFO:teuthology.orchestra.run.smithi109.stdout:NAME="Red Hat Enterprise Linux"
2022-05-02T09:33:54.543 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION="8.5 (Ootpa)"
2022-05-02T09:33:54.543 INFO:teuthology.orchestra.run.smithi109.stdout:ID="rhel"
2022-05-02T09:33:54.543 INFO:teuthology.orchestra.run.smithi109.stdout:ID_LIKE="fedora"
2022-05-02T09:33:54.543 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION_ID="8.5"
2022-05-02T09:33:54.543 INFO:teuthology.orchestra.run.smithi109.stdout:PLATFORM_ID="platform:el8"
2022-05-02T09:33:54.543 INFO:teuthology.orchestra.run.smithi109.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.5 (Ootpa)"
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:ANSI_COLOR="0;31"
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:HOME_URL="https://www.redhat.com/"
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.5
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-05-02T09:33:54.544 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.5"
2022-05-02T09:33:54.545 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server
2022-05-02T09:33:54.562 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server
2022-05-02T09:33:54.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com'
2022-05-02T09:33:54.768 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64
2022-05-02T09:33:54.827 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Red Hat Enterprise Linux"
2022-05-02T09:33:54.827 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="8.5 (Ootpa)"
2022-05-02T09:33:54.827 INFO:teuthology.orchestra.run.smithi124.stdout:ID="rhel"
2022-05-02T09:33:54.827 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="fedora"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="8.5"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el8"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.5 (Ootpa)"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.redhat.com/"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:
2022-05-02T09:33:54.828 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-05-02T09:33:54.829 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.5
2022-05-02T09:33:54.829 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-05-02T09:33:54.829 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.5"
2022-05-02T09:33:54.829 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server
2022-05-02T09:33:54.921 INFO:teuthology.dispatcher.supervisor:Running job 6818455
2022-05-02T09:33:54.921 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fd5c31669e51177d3c57bc0420279c5d1e944497/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-05-02_09:11:25-fs-wip-vshankar-testing1-20220428-204527-testing-default-smithi/6818455 --name vshankar-2022-05-02_09:11:25-fs-wip-vshankar-testing1-20220428-204527-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/recovery-fs} -- /home/teuthworker/archive/vshankar-2022-05-02_09:11:25-fs-wip-vshankar-testing1-20220428-204527-testing-default-smithi/6818455/orig.config.yaml
2022-05-02T09:33:54.924 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-05-02_09:11:25-fs-wip-vshankar-testing1-20220428-204527-testing-default-smithi/6818455
2022-05-02T09:33:54.925 INFO:teuthology.dispatcher.supervisor:Job PID: 28896
2022-05-02T09:33:54.925 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-05-02T16:04:23.990 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-05-02T16:04:24.190 INFO:teuthology.kill:Killing Pids: {28896}
2022-05-02T16:04:25.717 INFO:teuthology.task.internal:roles: ubuntu@smithi109.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2']
2022-05-02T16:04:25.717 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']
2022-05-02T16:04:25.718 INFO:teuthology.misc:Compressing logs...
2022-05-02T16:04:25.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com'
2022-05-02T16:04:25.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com'
2022-05-02T16:04:26.481 INFO:teuthology.misc:Compressing logs...
2022-05-02T16:04:26.792 INFO:teuthology.misc:Compressing logs...
2022-05-02T16:05:35.658 INFO:teuthology.kill:No teuthology processes running
2022-05-02T16:05:37.090 INFO:teuthology.kill:Nuking machines: ['smithi109', 'smithi124']
2022-05-02T16:05:38.298 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_fd5c31669e51177d3c57bc0420279c5d1e944497/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-05-02T16:05:38.318 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-05-02T16:05:38.318 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_fd5c31669e51177d3c57bc0420279c5d1e944497/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
2022-05-02T16:05:38.319 INFO:teuthology.kill:  "class": algorithms.Blowfish,
2022-05-02T16:05:38.319 INFO:teuthology.kill:2022-05-02 16:05:38,297.297 INFO:teuthology.nuke:targets:
2022-05-02T16:05:38.319 INFO:teuthology.kill:  smithi109.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwP+zm4vD3B0cTylZo0icjhrQ6+jERfLNsnL6qsiq2L7bN52Dyu5yu2Dh/sVuzesCeV9+LVyZb7GWDg9O7hLKhNPaFRahgX/T2iHFOi7sQQ/H/80xKJ/C9CsNFVCxepaRCPFSgRUCJeIXwAehPNNGDou6y7Yfu57wOxRdkbvOu2WMyYUrZoQf5qyxGLk54fEse+8V2sbuMPRzq45VRMRxrViZuXbyFuhH/8GurbMlBlv6GuRT2Lxw0SNrDDF0WJcdu8n3altYoZNl+iwrehm9rv95fp6r/gWSU8HxttNYUwqtsBt4yNFd2NApxI1QmopNtx8wKTuiYhqfae7VgZOR7qOQNI3/U6iFUiZVHiUgfXFsAohZ8SqUFRweE4cTCOeYf/OS8f4GJwFTQw3EUSCt3YEAIrYwi2CcWdHoY1HY4/wXxHSQTf46Qpt9PtaiYEhrGzb58oIUvL3sTDCGKVPRxx3mEoo0Ev0kzt1v6MeceCFtKuuji4DSXx5qWUG9Z77c=
2022-05-02T16:05:38.319 INFO:teuthology.kill:  smithi124.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMHWTCWEJ187vajUkojX2FMOX4FuuvFh7hsBch25gTomS+2LRJCytQAVm0XDd+GZfKIP6aAG8rn7Fy6GBxOwZXGv6tNPqhj75T88TU3vYVfuLgNMuZ/93hznwYL2lDLYMt/9xOILHM8CCIzpiJqi70pWF3cNPsWTbuuW6RmK6s0eBqt5AYNPIXpiPnLmoZJcv0acqDmGveoSIAg5/l76yRIZakTasgGR34ta3hoK3wPpX0T7QlWtf5sVafW924H2Cb9kxnZ0HbTNOb4dU1JpOT5BvJpRSsiDQf7UbBO0hol8112vakZvepNlNmefvA545yA1GMiiq1wlrfKdnXzb9Mzzd8PgSd3JrSTXqcG4siX/LBfVz/2jwpqDg8Bx6CwixstqWq01kxeDMwubgZ7vAjTyzOKfXXO69KJdWSOSwvGsbDH4rNkV9WrEz3gOUhuPiZ9UlY2S9YuXwXz0oO5vPyx/+OmtSNdSFy88on4JjYXJj49wwKKKGwj+yMl3MX8cU=
2022-05-02T16:05:38.330 INFO:teuthology.kill:2022-05-02 16:05:38,330.330 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-05-02T16:05:38.332 INFO:teuthology.kill:2022-05-02 16:05:38,332.332 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-05-02T16:05:38.412 INFO:teuthology.kill:2022-05-02 16:05:38,412.412 INFO:teuthology.orchestra.console:Power off smithi124
2022-05-02T16:05:38.427 INFO:teuthology.kill:2022-05-02 16:05:38,426.426 INFO:teuthology.orchestra.console:Power off smithi109
2022-05-02T16:05:50.900 INFO:teuthology.kill:2022-05-02 16:05:50,899.899 INFO:teuthology.orchestra.console:Power off for smithi124 completed
2022-05-02T16:05:50.904 INFO:teuthology.kill:2022-05-02 16:05:50,904.904 INFO:teuthology.orchestra.console:Power off for smithi109 completed
2022-05-02T16:05:51.048 INFO:teuthology.kill:2022-05-02 16:05:51,047.047 INFO:teuthology.lock.ops:unlocked: smithi109.front.sepia.ceph.com
2022-05-02T16:05:51.052 INFO:teuthology.kill:2022-05-02 16:05:51,052.052 INFO:teuthology.lock.ops:unlocked: smithi124.front.sepia.ceph.com
2022-05-02T16:07:51.399 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-05-02T16:07:51.525 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi109 but it was locked by another job: /home/teuthworker/archive/amathuri-2022-05-02_14:59:15-rados-wip-yuri3-testing-2022-04-22-0534-quincy-distro-default-smithi/6818608
2022-05-02T16:07:51.525 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi124 but it was locked by another job: /home/teuthworker/archive/amathuri-2022-05-02_14:59:15-rados-wip-yuri3-testing-2022-04-22-0534-quincy-distro-default-smithi/6818608