2021-11-11T07:26:04.558 INFO:root:teuthology version: 1.1.0-91815aeb 2021-11-11T07:26:04.681 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi064.front.sepia.ceph.com' reimaging 2021-11-11T07:26:04.683 INFO:teuthology.lock.ops:Updating [ubuntu@smithi064.front.sepia.ceph.com]: reset os type and version on server 2021-11-11T07:26:04.683 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2021-11-11T07:26:04.705 INFO:teuthology.lock.ops:Node 'ubuntu@smithi064.front.sepia.ceph.com' reimaging is complete 2021-11-11T07:26:04.705 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi073.front.sepia.ceph.com' reimaging 2021-11-11T07:26:04.706 INFO:teuthology.lock.ops:Updating [ubuntu@smithi073.front.sepia.ceph.com]: reset os type and version on server 2021-11-11T07:26:04.706 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2021-11-11T07:26:04.726 INFO:teuthology.lock.ops:Node 'ubuntu@smithi073.front.sepia.ceph.com' reimaging is complete 2021-11-11T07:26:04.897 INFO:teuthology.provision.fog.smithi064:Scheduling deploy of rhel 8.4 2021-11-11T07:26:04.928 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of rhel 8.4 2021-11-11T07:26:05.178 INFO:teuthology.orchestra.console:Power off smithi064 2021-11-11T07:26:05.217 INFO:teuthology.orchestra.console:Power off smithi073 2021-11-11T07:26:13.556 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2021-11-11T07:26:13.580 INFO:teuthology.orchestra.console:Power off for smithi073 completed 2021-11-11T07:26:13.658 INFO:teuthology.orchestra.console:Power on smithi064 2021-11-11T07:26:13.682 INFO:teuthology.orchestra.console:Power on smithi073 2021-11-11T07:26:17.928 INFO:teuthology.orchestra.console:Power on for smithi064 completed 2021-11-11T07:26:18.029 INFO:teuthology.provision.fog.smithi064:Waiting for deploy to finish 2021-11-11T07:26:22.048 INFO:teuthology.orchestra.console:Power on for smithi073 completed 2021-11-11T07:26:22.149 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish 2021-11-11T07:31:07.619 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-11T07:31:07.836 INFO:teuthology.provision.fog.smithi064:Node is ready 2021-11-11T07:31:07.852 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064.front.sepia.ceph.com 2021-11-11T07:31:07.908 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64 smithi064.front.sepia.ceph.com smithi064 2021-11-11T07:31:08.248 INFO:teuthology.provision.fog.smithi064:Deploy complete! 2021-11-11T07:31:11.564 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-11T07:31:11.770 INFO:teuthology.provision.fog.smithi073:Node is ready 2021-11-11T07:31:11.785 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073.front.sepia.ceph.com 2021-11-11T07:31:11.841 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73 smithi073.front.sepia.ceph.com smithi073 2021-11-11T07:31:12.167 INFO:teuthology.provision.fog.smithi073:Deploy complete! 2021-11-11T07:31:12.315 INFO:teuthology.lock.ops:Checking smithi064.front.sepia.ceph.com 2021-11-11T07:31:12.316 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-11T07:31:12.337 INFO:teuthology.lock.ops:Checking smithi073.front.sepia.ceph.com 2021-11-11T07:31:12.337 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-11T07:31:12.355 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: set os type and version on server 2021-11-11T07:31:12.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2021-11-11T07:31:12.594 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2021-11-11T07:31:12.651 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Red Hat Enterprise Linux" 2021-11-11T07:31:12.651 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8.4 (Ootpa)" 2021-11-11T07:31:12.652 INFO:teuthology.orchestra.run.smithi064.stdout:ID="rhel" 2021-11-11T07:31:12.652 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="fedora" 2021-11-11T07:31:12.652 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8.4" 2021-11-11T07:31:12.652 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2021-11-11T07:31:12.653 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-11-11T07:31:12.653 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2021-11-11T07:31:12.653 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-11-11T07:31:12.653 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.redhat.com/" 2021-11-11T07:31:12.653 INFO:teuthology.orchestra.run.smithi064.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-11-11T07:31:12.654 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-11T07:31:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: 2021-11-11T07:31:12.654 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-11T07:31:12.654 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-11-11T07:31:12.655 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-11-11T07:31:12.655 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-11-11T07:31:12.656 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2021-11-11T07:31:12.676 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: set os type and version on server 2021-11-11T07:31:12.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2021-11-11T07:31:12.898 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2021-11-11T07:31:12.952 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Red Hat Enterprise Linux" 2021-11-11T07:31:12.953 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8.4 (Ootpa)" 2021-11-11T07:31:12.953 INFO:teuthology.orchestra.run.smithi073.stdout:ID="rhel" 2021-11-11T07:31:12.953 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="fedora" 2021-11-11T07:31:12.953 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8.4" 2021-11-11T07:31:12.954 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8" 2021-11-11T07:31:12.954 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-11-11T07:31:12.954 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2021-11-11T07:31:12.954 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-11-11T07:31:12.954 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.redhat.com/" 2021-11-11T07:31:12.955 INFO:teuthology.orchestra.run.smithi073.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-11-11T07:31:12.955 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-11T07:31:12.955 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-11-11T07:31:12.955 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-11T07:31:12.956 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-11-11T07:31:12.956 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-11-11T07:31:12.956 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-11-11T07:31:12.956 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2021-11-11T07:31:13.051 INFO:teuthology.dispatcher.supervisor:Running job 6497156 2021-11-11T07:31:13.052 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_91815aeb94449f1ea4365867a0fd4c60103057e1/virtualenv/bin/teuthology -v --owner vshankar --archive /home/teuthworker/archive/vshankar-2021-11-11_07:20:45-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497156 --name vshankar-2021-11-11_07:20:45-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi --description fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} -- /home/teuthworker/archive/vshankar-2021-11-11_07:20:45-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497156/orig.config.yaml 2021-11-11T07:31:13.055 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2021-11-11_07:20:45-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497156 2021-11-11T07:31:13.056 INFO:teuthology.dispatcher.supervisor:Job PID: 32053 2021-11-11T07:31:13.056 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-11T19:32:07.491 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-11T19:32:07.648 INFO:teuthology.kill:Killing Pids: {32053} 2021-11-11T19:32:08.817 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-11-11T19:32:08.818 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2021-11-11T19:32:08.818 INFO:teuthology.misc:Compressing logs... 2021-11-11T19:32:08.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2021-11-11T19:32:09.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2021-11-11T19:32:09.401 INFO:teuthology.misc:Compressing logs... 2021-11-11T19:32:09.434 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory 2021-11-11T19:32:09.466 INFO:teuthology.orchestra.run.smithi073.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory 2021-11-11T19:32:09.503 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory 2021-11-11T19:32:09.504 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2021-11-11T19:32:09.531 INFO:teuthology.orchestra.run.smithi073.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory 2021-11-11T19:32:09.531 INFO:teuthology.orchestra.run.smithi073.stderr:tar: Error is not recoverable: exiting now 2021-11-11T19:32:09.533 INFO:teuthology.misc:Compressing logs... 2021-11-11T19:32:15.988 INFO:teuthology.kill:No teuthology processes running 2021-11-11T19:32:17.151 INFO:teuthology.kill:Nuking machines: ['smithi064', 'smithi073'] 2021-11-11T19:32:18.108 INFO:teuthology.kill:2021-11-11 19:32:18,108.108 INFO:teuthology.nuke:targets: 2021-11-11T19:32:18.109 INFO:teuthology.kill: smithi064.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5k/S5RtkyGr5EGLSfH1IPioIEpKwlkrUerOKDvpUi5S/GK0+6sTv5pPCi31SYJ95cMzQnoA//JluLpQqpRE8dxo/NJJ1meK9aLM0cDmTkVlIZo6HSFLggrLsITXVwQiQ+QW4vKLXUqvjr59cZABBVonYdCeYyKJP53w1q7dnbuI4H0T/sZ3TgT3WT/4q2gxz0hjwZeCogtc0DCfEoS6JMwnZr/tPWrFhHC1cLPLg8CnvUOqE4BfH6RRELhQHflFPEFaT7c4sf5n/CEKJ5jB/MGyEou2tYPgXsZRLCAE3VkVqtnbtlSUCSBcn9y6aSt2yRcuxPVaQe/NpMF3kKcDl7AN107JiExH9nRF5UkCAc6EzCI0L0LoXGyWw25VlFXHqX2G9EUR0k6EPHRKM/oin69kukjbnwTdPsF6c8ZbmfukqLnmJPVFD+ntm/9lL9WGovP9BMGWqvPxGDOAHEFz+RlDqITiiG3AEfytaf4WMGxARk6QNd6iJAz9CgVRyRJIE= 2021-11-11T19:32:18.109 INFO:teuthology.kill: smithi073.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMb9j9JRo1cKY/0uJX5tqaBVO6QYSr28tyUa6RCvKEvvWvJ06EHCUL2jJSmwtRoMDkd32z29a4G1XuzhEbhzkveqZm/A66Vr0I4A94afzIeJoCa2OavLm84bFJrvix3yQbn7KyRXByRSp6ExSEEYG7TxEn9u0glyMSTzaMSJxpGZYdr3uQTu+Ur+a3MKcAxrdrl4nZ0AIg8f9Pa4/nkpUy6mchl/8pIZbXRr4o3oHr5Llu7hF3Dmg62Ff8S53I/p49Lbac72FSBWgSX5WXZ4xgDXLU1EOt2NgGSX1Zh1YDjWtjRoSSJbi0SX1myefoAEUqESsSPsj3nt2Njbwu/Y3U6EDy5/jr5CCHJYvC6dqvhEfseTHkEZN/rwxlvREYeQNPt2+X+kl/qN2kY2F99z4jhWOkxdMluLKglL9JYNRvz0wMJ1vllwGGpp1mDwzpY4wlIVo5HafQ8ddBKROCN0aBxhJ9Gz6nzIGI35V0xQuU1owQKC28d5tN5tJ73i3sI48= 2021-11-11T19:32:18.136 INFO:teuthology.kill:2021-11-11 19:32:18,136.136 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-11T19:32:18.139 INFO:teuthology.kill:2021-11-11 19:32:18,139.139 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-11T19:32:18.215 INFO:teuthology.kill:2021-11-11 19:32:18,215.215 INFO:teuthology.orchestra.console:Power off smithi073 2021-11-11T19:32:18.228 INFO:teuthology.kill:2021-11-11 19:32:18,228.228 INFO:teuthology.orchestra.console:Power off smithi064 2021-11-11T19:32:26.586 INFO:teuthology.kill:2021-11-11 19:32:26,586.586 INFO:teuthology.orchestra.console:Power off for smithi073 completed 2021-11-11T19:32:26.732 INFO:teuthology.kill:2021-11-11 19:32:26,731.731 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com 2021-11-11T19:32:30.702 INFO:teuthology.kill:2021-11-11 19:32:30,702.702 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2021-11-11T19:32:30.858 INFO:teuthology.kill:2021-11-11 19:32:30,858.858 INFO:teuthology.lock.ops:unlocked: smithi064.front.sepia.ceph.com 2021-11-11T19:34:31.176 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-11T19:34:31.275 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi073 but it was locked by another job: /home/teuthworker/archive/ifed01-2021-11-11_17:08:08-rados-wip-ifed-faster-rm-p2-testing-distro-default-smithi/6497728