2022-05-31T18:20:34.913 INFO:root:teuthology version: 1.1.1.dev341+g1b30281f 2022-05-31T18:20:35.071 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging 2022-05-31T18:20:35.072 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server 2022-05-31T18:20:35.072 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2022-05-31T18:20:35.095 INFO:teuthology.lock.ops:Node 'smithi119.front.sepia.ceph.com' reimaging is complete 2022-05-31T18:20:35.095 INFO:teuthology.lock.ops:Start node 'smithi156.front.sepia.ceph.com' reimaging 2022-05-31T18:20:35.096 INFO:teuthology.lock.ops:Updating [smithi156.front.sepia.ceph.com]: reset os type and version on server 2022-05-31T18:20:35.096 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2022-05-31T18:20:35.120 INFO:teuthology.lock.ops:Node 'smithi156.front.sepia.ceph.com' reimaging is complete 2022-05-31T18:20:35.304 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of rhel 8.5 2022-05-31T18:20:35.324 INFO:teuthology.provision.fog.smithi156:Scheduling deploy of rhel 8.5 2022-05-31T18:20:35.594 INFO:teuthology.orchestra.console:Power off smithi119 2022-05-31T18:20:35.636 INFO:teuthology.orchestra.console:Power off smithi156 2022-05-31T18:20:43.971 INFO:teuthology.orchestra.console:Power off for smithi119 completed 2022-05-31T18:20:44.004 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2022-05-31T18:20:44.073 INFO:teuthology.orchestra.console:Power on smithi119 2022-05-31T18:20:44.106 INFO:teuthology.orchestra.console:Power on smithi156 2022-05-31T18:20:52.444 INFO:teuthology.orchestra.console:Power on for smithi119 completed 2022-05-31T18:20:52.472 INFO:teuthology.orchestra.console:Power on for smithi156 completed 2022-05-31T18:20:52.546 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish 2022-05-31T18:20:52.573 INFO:teuthology.provision.fog.smithi156:Waiting for deploy to finish 2022-05-31T18:23:23.927 ERROR:teuthology.orchestra.connection:Error authenticating with smithi119.front.sepia.ceph.com: Authentication failed. 2022-05-31T18:25:11.287 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-31T18:25:36.780 INFO:teuthology.provision.fog.smithi156:Node is ready 2022-05-31T18:25:36.803 INFO:teuthology.orchestra.run.smithi156.stdout:smithi156.front.sepia.ceph.com 2022-05-31T18:25:36.862 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156 smithi156.front.sepia.ceph.com smithi156 2022-05-31T18:25:37.242 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-31T18:25:37.245 INFO:teuthology.provision.fog.smithi156:Deploy complete! 2022-05-31T18:25:37.491 INFO:teuthology.provision.fog.smithi119:Node is ready 2022-05-31T18:25:37.507 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119.front.sepia.ceph.com 2022-05-31T18:25:37.564 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119 smithi119.front.sepia.ceph.com smithi119 2022-05-31T18:25:37.886 INFO:teuthology.provision.fog.smithi119:Deploy complete! 2022-05-31T18:25:38.126 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com 2022-05-31T18:25:38.126 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-31T18:25:38.151 INFO:teuthology.lock.ops:Checking smithi156.front.sepia.ceph.com 2022-05-31T18:25:38.151 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-31T18:25:38.170 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server 2022-05-31T18:25:38.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2022-05-31T18:25:38.363 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2022-05-31T18:25:38.418 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Red Hat Enterprise Linux" 2022-05-31T18:25:38.419 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8.5 (Ootpa)" 2022-05-31T18:25:38.419 INFO:teuthology.orchestra.run.smithi119.stdout:ID="rhel" 2022-05-31T18:25:38.419 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="fedora" 2022-05-31T18:25:38.419 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8.5" 2022-05-31T18:25:38.419 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8" 2022-05-31T18:25:38.419 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.5 (Ootpa)" 2022-05-31T18:25:38.419 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2022-05-31T18:25:38.419 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-05-31T18:25:38.420 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.redhat.com/" 2022-05-31T18:25:38.420 INFO:teuthology.orchestra.run.smithi119.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-05-31T18:25:38.420 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-31T18:25:38.420 INFO:teuthology.orchestra.run.smithi119.stdout: 2022-05-31T18:25:38.420 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-31T18:25:38.420 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.5 2022-05-31T18:25:38.420 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-05-31T18:25:38.420 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.5" 2022-05-31T18:25:38.421 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2022-05-31T18:25:38.442 INFO:teuthology.lock.ops:Updating [smithi156.front.sepia.ceph.com]: set os type and version on server 2022-05-31T18:25:38.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2022-05-31T18:25:38.657 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2022-05-31T18:25:38.721 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2022-05-31T18:25:38.722 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.5 (Ootpa)" 2022-05-31T18:25:38.722 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2022-05-31T18:25:38.722 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2022-05-31T18:25:38.722 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.5" 2022-05-31T18:25:38.722 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2022-05-31T18:25:38.722 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.5 (Ootpa)" 2022-05-31T18:25:38.722 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2022-05-31T18:25:38.722 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-05-31T18:25:38.723 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2022-05-31T18:25:38.723 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-05-31T18:25:38.723 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-31T18:25:38.723 INFO:teuthology.orchestra.run.smithi156.stdout: 2022-05-31T18:25:38.723 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-31T18:25:38.723 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.5 2022-05-31T18:25:38.723 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-05-31T18:25:38.723 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.5" 2022-05-31T18:25:38.724 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2022-05-31T18:25:38.825 INFO:teuthology.dispatcher.supervisor:Running job 6855637 2022-05-31T18:25:38.826 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_1b30281f276b97a8186594b7f92fe1f728418ada/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2022-05-31_14:46:01-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6855637 --name rishabh-2022-05-31_14:46:01-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} -- /home/teuthworker/archive/rishabh-2022-05-31_14:46:01-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6855637/orig.config.yaml 2022-05-31T18:25:38.830 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2022-05-31_14:46:01-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6855637 2022-05-31T18:25:38.831 INFO:teuthology.dispatcher.supervisor:Job PID: 28564 2022-05-31T18:25:38.831 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-06-01T00:55:51.842 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-06-01T00:55:52.022 INFO:teuthology.kill:Killing Pids: {28564} 2022-06-01T00:55:53.568 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-06-01T00:55:53.568 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-06-01T00:55:53.568 INFO:teuthology.misc:Compressing logs... 2022-06-01T00:55:53.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2022-06-01T00:55:53.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2022-06-01T00:55:54.200 INFO:teuthology.misc:Compressing logs... 2022-06-01T00:55:54.440 INFO:teuthology.misc:Compressing logs... 2022-06-01T00:56:47.847 INFO:teuthology.kill:No teuthology processes running 2022-06-01T00:56:49.304 INFO:teuthology.kill:Nuking machines: ['smithi119', 'smithi156'] 2022-06-01T00:56:50.575 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_1b30281f276b97a8186594b7f92fe1f728418ada/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-06-01T00:56:50.601 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-06-01T00:56:50.601 INFO:teuthology.kill:2022-06-01 00:56:50,575.575 INFO:teuthology.nuke:targets: 2022-06-01T00:56:50.601 INFO:teuthology.kill: smithi119.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoNAdL1SUTk4IEcuFjTNywLe8qpWPDQEDInUjw+FvzxMSwgwtKCyqWo83qP0599s298YprPXfVCWbYhGVYoy2CkdWy6jr2pIE6TScrUecTynshRVsCbGp7ezN1h3iEebqOUCAoyS1vO4uPrsgxscYFzYMM5ImCe7bVI2zS5pgDFs0QM22JJ9eSRyioBInrfcVZYvdMC3miXXm2oawwDSNT4D2RCyPVoiZQhs8VglmrlGo8bmmec6u2ifUz9QHqhdiWL9dByLMBfWnrMFGUJdtwJ385TTwwoAPZpqlbBzdTnggjbNyFa6s0P9op5F5u/RzBm7PVC2g8mF83LmYeHVpaV5Nsl2vHChGlZphw1r4N1vwGBgbbej/vN5XJiIbOF9Zse8AiWpSGVeRpEH6J0f/1UyR8dBOKqgDJ5yO4JTl8kbzE/uYM8kaL6DvwiT1Vys/yTruARqGuvS0qgMhVvKty9WLpmi85CNFxs3NMUNZ/Hov2v0me3ZcGg88rGrvUZ+0= 2022-06-01T00:56:50.601 INFO:teuthology.kill: smithi156.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWaK/hwfBDSyU+zlKC+P6dN/kfrC+IyaKfqq8E436utOTnL/xudIT8bUaP+2PG8U9IcJpXjejTK4oMD8Ev1hNCWmzm12GsEoNaOle6G5oEj+GQa3UIMgvgqGzMJgVprRX+fIaf0338vDD66OlywrDr05RLSnS2JU3yl73JnTLMXANFYh6x54NsHHPmk3dFYPq6DPFs6OWrpCaLF2QZdMq96VCVYhoZB2r5NaAcq6vqCT/j7M1E1Ia0hWoQ0Dvqn4CW0n5wOHkubYHs0RuyIbiI9pvD5J2rwCNTe2VDBoXLsVhYmW1jDzmzJoysOTZ64ia9/ITC7op7DfacNGhssMXf3NktbTeoetmR2Mvy7lQWFdoayyZkSbIdyRoyfinO8JcKevtlgrZMucfNIvIiobpvCFwwWpxktlpLL9lGyjBd2hgMXB8sT6KXigk9sGRH2/BD34lemRpqee3D5T1fZKd2FS3+5+et02bW+b6eDN8P7TQjatzOFstqXWBbfpk8Fck= 2022-06-01T00:56:50.607 INFO:teuthology.kill:2022-06-01 00:56:50,607.607 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-01T00:56:50.611 INFO:teuthology.kill:2022-06-01 00:56:50,611.611 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-01T00:56:50.698 INFO:teuthology.kill:2022-06-01 00:56:50,697.697 INFO:teuthology.orchestra.console:Power off smithi119 2022-06-01T00:56:50.715 INFO:teuthology.kill:2022-06-01 00:56:50,714.714 INFO:teuthology.orchestra.console:Power off smithi156 2022-06-01T00:56:59.069 INFO:teuthology.kill:2022-06-01 00:56:59,069.069 INFO:teuthology.orchestra.console:Power off for smithi119 completed 2022-06-01T00:56:59.084 INFO:teuthology.kill:2022-06-01 00:56:59,084.084 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2022-06-01T00:56:59.243 INFO:teuthology.kill:2022-06-01 00:56:59,241.241 INFO:teuthology.lock.ops:unlocked: smithi156.front.sepia.ceph.com 2022-06-01T00:56:59.249 INFO:teuthology.kill:2022-06-01 00:56:59,248.248 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com 2022-06-01T00:58:59.583 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-06-01T00:58:59.701 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi119 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-05-31_21:35:41-rados-wip-yuri2-testing-2022-05-31-1300-pacific-distro-default-smithi/6856556 2022-06-01T00:58:59.702 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi156 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-05-31_21:35:41-rados-wip-yuri2-testing-2022-05-31-1300-pacific-distro-default-smithi/6856556