2021-11-11T16:26:04.670 INFO:root:teuthology version: 1.1.0-91815aeb 2021-11-11T16:26:04.793 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi152.front.sepia.ceph.com' reimaging 2021-11-11T16:26:04.794 INFO:teuthology.lock.ops:Updating [ubuntu@smithi152.front.sepia.ceph.com]: reset os type and version on server 2021-11-11T16:26:04.794 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2021-11-11T16:26:04.814 INFO:teuthology.lock.ops:Node 'ubuntu@smithi152.front.sepia.ceph.com' reimaging is complete 2021-11-11T16:26:04.814 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi188.front.sepia.ceph.com' reimaging 2021-11-11T16:26:04.814 INFO:teuthology.lock.ops:Updating [ubuntu@smithi188.front.sepia.ceph.com]: reset os type and version on server 2021-11-11T16:26:04.815 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2021-11-11T16:26:04.836 INFO:teuthology.lock.ops:Node 'ubuntu@smithi188.front.sepia.ceph.com' reimaging is complete 2021-11-11T16:26:05.013 INFO:teuthology.provision.fog.smithi152:Scheduling deploy of rhel 8.4 2021-11-11T16:26:05.032 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of rhel 8.4 2021-11-11T16:26:05.313 INFO:teuthology.orchestra.console:Power off smithi152 2021-11-11T16:26:05.353 INFO:teuthology.orchestra.console:Power off smithi188 2021-11-11T16:26:13.690 INFO:teuthology.orchestra.console:Power off for smithi152 completed 2021-11-11T16:26:13.717 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2021-11-11T16:26:13.792 INFO:teuthology.orchestra.console:Power on smithi152 2021-11-11T16:26:13.820 INFO:teuthology.orchestra.console:Power on smithi188 2021-11-11T16:26:22.161 INFO:teuthology.orchestra.console:Power on for smithi152 completed 2021-11-11T16:26:22.179 INFO:teuthology.orchestra.console:Power on for smithi188 completed 2021-11-11T16:26:22.263 INFO:teuthology.provision.fog.smithi152:Waiting for deploy to finish 2021-11-11T16:26:22.280 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish 2021-11-11T16:28:53.348 ERROR:teuthology.orchestra.connection:Error authenticating with smithi188.front.sepia.ceph.com: Authentication failed. 2021-11-11T16:31:06.627 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-11T16:31:06.834 INFO:teuthology.provision.fog.smithi188:Node is ready 2021-11-11T16:31:06.849 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188.front.sepia.ceph.com 2021-11-11T16:31:06.905 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188 2021-11-11T16:31:07.226 INFO:teuthology.provision.fog.smithi188:Deploy complete! 2021-11-11T16:31:26.761 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-11T16:31:26.979 INFO:teuthology.provision.fog.smithi152:Node is ready 2021-11-11T16:31:26.994 INFO:teuthology.orchestra.run.smithi152.stdout:smithi152.front.sepia.ceph.com 2021-11-11T16:31:27.049 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152 smithi152.front.sepia.ceph.com smithi152 2021-11-11T16:31:27.387 INFO:teuthology.provision.fog.smithi152:Deploy complete! 2021-11-11T16:31:27.543 INFO:teuthology.lock.ops:Checking smithi152.front.sepia.ceph.com 2021-11-11T16:31:27.544 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-11T16:31:27.564 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com 2021-11-11T16:31:27.565 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-11T16:31:27.584 INFO:teuthology.lock.ops:Updating [smithi152.front.sepia.ceph.com]: set os type and version on server 2021-11-11T16:31:27.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2021-11-11T16:31:27.766 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2021-11-11T16:31:27.820 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="Red Hat Enterprise Linux" 2021-11-11T16:31:27.820 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="8.4 (Ootpa)" 2021-11-11T16:31:27.820 INFO:teuthology.orchestra.run.smithi152.stdout:ID="rhel" 2021-11-11T16:31:27.821 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="fedora" 2021-11-11T16:31:27.821 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="8.4" 2021-11-11T16:31:27.821 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el8" 2021-11-11T16:31:27.821 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-11-11T16:31:27.822 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2021-11-11T16:31:27.822 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-11-11T16:31:27.822 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://www.redhat.com/" 2021-11-11T16:31:27.823 INFO:teuthology.orchestra.run.smithi152.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-11-11T16:31:27.823 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-11T16:31:27.824 INFO:teuthology.orchestra.run.smithi152.stdout: 2021-11-11T16:31:27.824 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-11T16:31:27.825 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-11-11T16:31:27.825 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-11-11T16:31:27.825 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-11-11T16:31:27.828 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2021-11-11T16:31:27.848 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server 2021-11-11T16:31:27.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2021-11-11T16:31:28.086 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2021-11-11T16:31:28.141 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="Red Hat Enterprise Linux" 2021-11-11T16:31:28.141 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8.4 (Ootpa)" 2021-11-11T16:31:28.141 INFO:teuthology.orchestra.run.smithi188.stdout:ID="rhel" 2021-11-11T16:31:28.142 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="fedora" 2021-11-11T16:31:28.142 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8.4" 2021-11-11T16:31:28.142 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8" 2021-11-11T16:31:28.142 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-11-11T16:31:28.143 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2021-11-11T16:31:28.143 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-11-11T16:31:28.143 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://www.redhat.com/" 2021-11-11T16:31:28.143 INFO:teuthology.orchestra.run.smithi188.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-11-11T16:31:28.143 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-11T16:31:28.144 INFO:teuthology.orchestra.run.smithi188.stdout: 2021-11-11T16:31:28.144 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-11T16:31:28.144 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-11-11T16:31:28.144 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-11-11T16:31:28.144 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-11-11T16:31:28.145 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2021-11-11T16:31:28.242 INFO:teuthology.dispatcher.supervisor:Running job 6497679 2021-11-11T16:31:28.242 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_91815aeb94449f1ea4365867a0fd4c60103057e1/virtualenv/bin/teuthology -v --owner vshankar --archive /home/teuthworker/archive/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497679 --name vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi --description fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} -- /home/teuthworker/archive/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497679/orig.config.yaml 2021-11-11T16:31:28.246 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497679 2021-11-11T16:31:28.247 INFO:teuthology.dispatcher.supervisor:Job PID: 8862 2021-11-11T16:31:28.247 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-12T04:31:48.477 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-12T04:31:48.685 INFO:teuthology.kill:Killing Pids: {8862} 2021-11-12T04:31:50.785 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-11-12T04:31:50.788 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2021-11-12T04:31:50.788 INFO:teuthology.misc:Compressing logs... 2021-11-12T04:31:50.789 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2021-11-12T04:31:51.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2021-11-12T04:31:51.381 INFO:teuthology.misc:Compressing logs... 2021-11-12T04:31:51.409 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory 2021-11-12T04:31:51.445 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory 2021-11-12T04:31:51.474 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory 2021-11-12T04:31:51.475 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2021-11-12T04:31:51.510 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory 2021-11-12T04:31:51.510 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2021-11-12T04:31:51.511 INFO:teuthology.misc:Compressing logs... 2021-11-12T04:32:03.243 INFO:teuthology.kill:No teuthology processes running 2021-11-12T04:32:04.402 INFO:teuthology.kill:Nuking machines: ['smithi152', 'smithi188'] 2021-11-12T04:32:05.515 INFO:teuthology.kill:2021-11-12 04:32:05,514.514 INFO:teuthology.nuke:targets: 2021-11-12T04:32:05.928 INFO:teuthology.kill: smithi152.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpZuFP4TQTOZjgKZdi9nuVFfrdvfn9yO3AsbAlFcwiwIMV25yEtc29M06U9R+HnE9krxQVOCu8ToORuTP3soiLo0bIZj+XKyh5S6jjLy9opqA5VGCz8US4XhKCjSYtf/Xp+NLOJF0bNOhsn9sVFh4kbLQvlZPtfA+UGl1CzJ+wuzPK0STidmbubWR2UN0GEDZcLea5al5TQhj+ElG4xVjxaitDVSpzQ6YBVMg2klIbCbQu2Qqj1cYOWwuCNstqW6+RXWQK2x890CqjiBVni/Baippvw/7LLlkglR/9GMkHDYQFKNn5SLoxbA1x2qLuKpZZ/9EpjpBCdQawPU2mwECzUnmKTAn5r3FYd3utKve7+/hFbVWxgAsvZaHsDShwmuaheVeaUsyHb3+HyT7wk85bnjIChfkUkgvaHpHkXrFwA82pVisERG/mUIZ+KlbtIc7TrA0Fr2BeSUUC8wiFAkRydE3zNWGQmCj/OcSam/O2HjYkkEzdU+KafiRNKUMwjH8= 2021-11-12T04:32:05.928 INFO:teuthology.kill: smithi188.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfQv6j+r9nITN6fNcdQ9+VgnzPWbAupJpw3pzD3JhnHTy6NrK0GaD1ugWis3jjWxjOFb1yEJaeXTsxrMtmWh+X+kOsD7316vrAJyte9e4ekt1z8XEslxwrHepVbOPMgzIsD3xpRUG3tnsDNLu6wKehsLXMqBDKJWwMfnoq8daRj+ZyD9PC93+9xni0Yx/9uz2y+AtfTTkYgBfK22KmQNd9xDqU3KZJcCvIh+SuLt09WIuOEo5oOxepGBZLiigR8+278HOjWQUrNWwIGvhU2+hcasLyEwulwQRh6s/WDrYG6qgnJUf/7C/qH5eeCx7yIb4wBlJzQJUE7jPdb0OSLVgHgk/Q4SJ6eL9qzgamQN4N0820WY6FzT/6RIk/putx6MoHaTO7BaX+Ux8kdSKBlOADofCIZJlo0g8m4PuOZfNwc+mEuJZNH7Qip3BFkw+kXt1NH29CqH0SDdXTc0Llm4oLqu49fMtyKDw4LtMbpLhsDVASAFy8RpGG7cvWrAruZSk= 2021-11-12T04:32:05.929 INFO:teuthology.kill:2021-11-12 04:32:05,550.550 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-12T04:32:05.929 INFO:teuthology.kill:2021-11-12 04:32:05,554.554 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-12T04:32:05.929 INFO:teuthology.kill:2021-11-12 04:32:05,642.642 INFO:teuthology.orchestra.console:Power off smithi188 2021-11-12T04:32:05.930 INFO:teuthology.kill:2021-11-12 04:32:05,660.660 INFO:teuthology.orchestra.console:Power off smithi152 2021-11-12T04:32:18.149 INFO:teuthology.kill:2021-11-12 04:32:18,148.148 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2021-11-12T04:32:18.155 INFO:teuthology.kill:2021-11-12 04:32:18,155.155 INFO:teuthology.orchestra.console:Power off for smithi152 completed 2021-11-12T04:32:18.308 INFO:teuthology.kill:2021-11-12 04:32:18,307.307 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com 2021-11-12T04:32:18.311 INFO:teuthology.kill:2021-11-12 04:32:18,310.310 INFO:teuthology.lock.ops:unlocked: smithi152.front.sepia.ceph.com 2021-11-12T04:34:18.532 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-12T04:34:18.638 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi152 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi/6498303 2021-11-12T04:34:18.638 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi188 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi/6498303