2021-12-04T13:15:42.828 INFO:root:teuthology version: 1.1.0-844d1377
2021-12-04T13:15:42.977 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging
2021-12-04T13:15:42.978 INFO:teuthology.lock.ops:Updating [ubuntu@smithi134.front.sepia.ceph.com]: reset os type and version on server
2021-12-04T13:15:42.979 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server
2021-12-04T13:15:42.996 INFO:teuthology.lock.ops:Node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging is complete
2021-12-04T13:15:42.996 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi143.front.sepia.ceph.com' reimaging
2021-12-04T13:15:42.998 INFO:teuthology.lock.ops:Updating [ubuntu@smithi143.front.sepia.ceph.com]: reset os type and version on server
2021-12-04T13:15:42.998 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server
2021-12-04T13:15:43.017 INFO:teuthology.lock.ops:Node 'ubuntu@smithi143.front.sepia.ceph.com' reimaging is complete
2021-12-04T13:15:43.017 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi195.front.sepia.ceph.com' reimaging
2021-12-04T13:15:43.018 INFO:teuthology.lock.ops:Updating [ubuntu@smithi195.front.sepia.ceph.com]: reset os type and version on server
2021-12-04T13:15:43.018 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server
2021-12-04T13:15:43.037 INFO:teuthology.lock.ops:Node 'ubuntu@smithi195.front.sepia.ceph.com' reimaging is complete
2021-12-04T13:15:43.241 INFO:teuthology.provision.fog.smithi195:Scheduling deploy of ubuntu 20.04
2021-12-04T13:15:43.271 INFO:teuthology.provision.fog.smithi134:Scheduling deploy of ubuntu 20.04
2021-12-04T13:15:43.298 INFO:teuthology.provision.fog.smithi143:Scheduling deploy of ubuntu 20.04
2021-12-04T13:15:43.722 INFO:teuthology.orchestra.console:Power off smithi195
2021-12-04T13:15:43.738 INFO:teuthology.orchestra.console:Power off smithi134
2021-12-04T13:15:43.815 INFO:teuthology.orchestra.console:Power off smithi143
2021-12-04T13:15:43.879 INFO:teuthology.orchestra.console:Power off for smithi134 completed
2021-12-04T13:15:43.980 INFO:teuthology.orchestra.console:Power on smithi134
2021-12-04T13:15:48.236 INFO:teuthology.orchestra.console:Power on for smithi134 completed
2021-12-04T13:15:48.337 INFO:teuthology.provision.fog.smithi134:Waiting for deploy to finish
2021-12-04T13:15:52.181 INFO:teuthology.orchestra.console:Power off for smithi143 completed
2021-12-04T13:15:52.283 INFO:teuthology.orchestra.console:Power on smithi143
2021-12-04T13:15:56.208 INFO:teuthology.orchestra.console:Power off for smithi195 completed
2021-12-04T13:15:56.309 INFO:teuthology.orchestra.console:Power on smithi195
2021-12-04T13:16:00.643 INFO:teuthology.orchestra.console:Power on for smithi143 completed
2021-12-04T13:16:00.744 INFO:teuthology.provision.fog.smithi143:Waiting for deploy to finish
2021-12-04T13:16:04.674 INFO:teuthology.orchestra.console:Power on for smithi195 completed
2021-12-04T13:16:04.775 INFO:teuthology.provision.fog.smithi195:Waiting for deploy to finish
2021-12-04T13:19:09.214 ERROR:teuthology.orchestra.connection:Error authenticating with smithi195.front.sepia.ceph.com: Authentication failed.
2021-12-04T13:21:31.485 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-04T13:21:32.025 INFO:teuthology.provision.fog.smithi134:Node is ready
2021-12-04T13:21:32.032 INFO:teuthology.orchestra.run.smithi134.stdout:smithi134.front.sepia.ceph.com
2021-12-04T13:21:32.039 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.15.134 smithi134.front.sepia.ceph.com smithi134
2021-12-04T13:21:32.259 INFO:teuthology.provision.fog.smithi134:Deploy complete!
2021-12-04T13:21:41.037 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-04T13:21:41.628 INFO:teuthology.provision.fog.smithi195:Node is ready
2021-12-04T13:21:41.635 INFO:teuthology.orchestra.run.smithi195.stdout:smithi195.front.sepia.ceph.com
2021-12-04T13:21:41.641 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.15.195 smithi195.front.sepia.ceph.com smithi195
2021-12-04T13:21:41.848 INFO:teuthology.provision.fog.smithi195:Deploy complete!
2021-12-04T13:22:24.137 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-04T13:22:24.633 INFO:teuthology.provision.fog.smithi143:Node is ready
2021-12-04T13:22:24.640 INFO:teuthology.orchestra.run.smithi143.stdout:smithi143.front.sepia.ceph.com
2021-12-04T13:22:24.647 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143 smithi143.front.sepia.ceph.com smithi143
2021-12-04T13:22:24.879 INFO:teuthology.provision.fog.smithi143:Deploy complete!
2021-12-04T13:22:25.099 INFO:teuthology.lock.ops:Checking smithi134.front.sepia.ceph.com
2021-12-04T13:22:25.100 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-04T13:22:25.122 INFO:teuthology.lock.ops:Checking smithi143.front.sepia.ceph.com
2021-12-04T13:22:25.123 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-04T13:22:25.140 INFO:teuthology.lock.ops:Checking smithi195.front.sepia.ceph.com
2021-12-04T13:22:25.140 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-04T13:22:25.159 INFO:teuthology.lock.ops:Updating [smithi134.front.sepia.ceph.com]: set os type and version on server
2021-12-04T13:22:25.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com'
2021-12-04T13:22:25.334 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64
2021-12-04T13:22:25.384 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Ubuntu"
2021-12-04T13:22:25.384 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-12-04T13:22:25.385 INFO:teuthology.orchestra.run.smithi134.stdout:ID=ubuntu
2021-12-04T13:22:25.385 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE=debian
2021-12-04T13:22:25.385 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-12-04T13:22:25.385 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="20.04"
2021-12-04T13:22:25.386 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.ubuntu.com/"
2021-12-04T13:22:25.386 INFO:teuthology.orchestra.run.smithi134.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-12-04T13:22:25.386 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-12-04T13:22:25.386 INFO:teuthology.orchestra.run.smithi134.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-12-04T13:22:25.386 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_CODENAME=focal
2021-12-04T13:22:25.387 INFO:teuthology.orchestra.run.smithi134.stdout:UBUNTU_CODENAME=focal
2021-12-04T13:22:25.388 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server
2021-12-04T13:22:25.408 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: set os type and version on server
2021-12-04T13:22:25.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com'
2021-12-04T13:22:25.626 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64
2021-12-04T13:22:25.632 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Ubuntu"
2021-12-04T13:22:25.632 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-12-04T13:22:25.633 INFO:teuthology.orchestra.run.smithi143.stdout:ID=ubuntu
2021-12-04T13:22:25.633 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE=debian
2021-12-04T13:22:25.633 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-12-04T13:22:25.633 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="20.04"
2021-12-04T13:22:25.634 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.ubuntu.com/"
2021-12-04T13:22:25.634 INFO:teuthology.orchestra.run.smithi143.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-12-04T13:22:25.634 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-12-04T13:22:25.634 INFO:teuthology.orchestra.run.smithi143.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-12-04T13:22:25.635 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_CODENAME=focal
2021-12-04T13:22:25.635 INFO:teuthology.orchestra.run.smithi143.stdout:UBUNTU_CODENAME=focal
2021-12-04T13:22:25.635 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server
2021-12-04T13:22:25.654 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: set os type and version on server
2021-12-04T13:22:25.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com'
2021-12-04T13:22:25.922 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64
2021-12-04T13:22:25.929 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu"
2021-12-04T13:22:25.929 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-12-04T13:22:25.929 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu
2021-12-04T13:22:25.930 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian
2021-12-04T13:22:25.930 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-12-04T13:22:25.930 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="20.04"
2021-12-04T13:22:25.930 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/"
2021-12-04T13:22:25.931 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-12-04T13:22:25.931 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-12-04T13:22:25.931 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-12-04T13:22:25.931 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=focal
2021-12-04T13:22:25.931 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=focal
2021-12-04T13:22:25.932 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server
2021-12-04T13:22:26.024 INFO:teuthology.dispatcher.supervisor:Running job 6544509
2021-12-04T13:22:26.025 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-12-04_13:15:03-multimds-octopus-distro-default-smithi/6544509 --name teuthology-2021-12-04_13:15:03-multimds-octopus-distro-default-smithi --description multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} -- /home/teuthworker/archive/teuthology-2021-12-04_13:15:03-multimds-octopus-distro-default-smithi/6544509/orig.config.yaml
2021-12-04T13:22:26.028 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-12-04_13:15:03-multimds-octopus-distro-default-smithi/6544509
2021-12-04T13:22:26.028 INFO:teuthology.dispatcher.supervisor:Job PID: 20114
2021-12-04T13:22:26.029 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-05T01:22:47.386 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-05T01:22:47.582 INFO:teuthology.kill:Killing Pids: {20114}
2021-12-05T01:22:48.811 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3']
2021-12-05T01:22:48.812 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'mds.b', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-12-05T01:22:48.812 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['client.0', 'client.1']
2021-12-05T01:22:48.813 INFO:teuthology.misc:Compressing logs...
2021-12-05T01:22:48.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com'
2021-12-05T01:22:48.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com'
2021-12-05T01:22:49.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com'
2021-12-05T01:22:49.836 INFO:teuthology.kill:No teuthology processes running
2021-12-05T01:22:50.883 INFO:teuthology.kill:Nuking machines: ['smithi134', 'smithi143', 'smithi195']
2021-12-05T01:22:51.824 INFO:teuthology.kill:2021-12-05 01:22:51,824.824 INFO:teuthology.nuke:targets:
2021-12-05T01:22:51.825 INFO:teuthology.kill:  smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKgfFMO0QF3hdRzoyKtDPfPUHAMLfAHULB4m7dpWTqELJoQaK4Add71WA0w3nNu6jIwls8oMb4IqKMM+ZgfinBpzPfZTyifvmP/MVSIkvc+fmX/aAtgAZKbrmej671WtKEQof9uT8bKigJ3GAR35hDsekdwlB2wLkahMJx/ebXpzlOZVEuy40NlIq5Nsrd7z/CZqJBOr2newNiCJTXbx1TfmqvJlz3CNSZ0aBzw7UINzWbLVQs47h0zjzxFh6zhcY8whmpdXKSBgXluUmMm+l3TrdgY4llCYBu5iHU6QbfQbeODHndZa8tTePi+hU/1dNE0uT8fUcsVWGFghrUxYykFZd2osMeS5hk/bu+cH5ZlYLJ4lcBY/IkqnaMLea3LvsV6Oa3Ljtcz0gn9Bg/7WiVUbeXTyH3xVyl8p32hHrmQpKOApYlt8e5FazZwXIPlI/Lf5u79B9HxXu2afbocXcH4I8mWFH7c7aCB5vIQEl96/CUawnCap82ASUTmgNqXCs=
2021-12-05T01:22:51.825 INFO:teuthology.kill:  smithi143.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf27HeHcbqxmCqIdI7Zdt2XzCmRLkgLwvXNTIaEjlCFW+in1RlPgeuZbGtPgHjK2FCMckUaeNrgbegYTPrYxLvf5y/sFN38Zrddiv0kxZzlzrqv5H4CLciH0Jtsa7NYJ8zLTPg/m4+e2ZZbc6fwDR5NuizWN0gPpYCFtP7boyas4FOxBDMYUZoVTsR0h1Ps+Q/6UJ2J/KJcdBQ+hMlkTjv/EP7dWf4WMQyUnQWs0tL4negHuboEg4QhI5QvaRzuIXObd3jwbcmRwya6w5jRrhoiDgCUjSd10mI3+mCZrFYo3osxejg6sU50ZMqKCSRZexFflg0bW5tzBESHd3i0zE9XkBgHXbd3DNxLrjaJfkSwmsgLVZBLNkHz21M0E/5opG7IaVdyqlCo2xqaUO4PsC1Out9taXSIrEqyy3qmob4rdcxVX6imtjDLLnn3mPvSLPJTyt5VFLkNj2Kq7izJlMbxKWBeDFsHUPpFdt5vqeQnu1laD75wLky49ZZumOyNsU=
2021-12-05T01:22:51.826 INFO:teuthology.kill:  smithi195.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1sE4G7LhpL5L/cEfQGBsp1Y1Gv9UemCtfeuPl9aqF/7k67jbNgaBXCXO/sEOSE7GDrYJ6VQdixP8AH1cGIvXcD06YO8KQv6KOkxal72p5h3V5y8kZW8MLkA0/Ty1aAAsqar7V5NOIyO6aBm3QIvjdTgJiEygNuFnHUSUd9yUHxXChOvWwGi/kx/4ElcolYEgGKHqVVqXk0IBn7cHeeYj1lnDxYTrTuIW7KyVQ2753HMlGpMltThjei/s9hJsjcQNsXF2E4bIEy/oikaRe6VirshFhEvVUaqajLA12Fgacd/T/NjZdexGi9l/E5YxMD5Avf3+DZaSoVC10NR/cOfAKdrHIWm3XMIhEirq1LRXAQDx4UwvYnLazKoXlWoM+N108WssP+ZL1+ioIzjcUP4v1EaD0wSooG6NWkrBqs4Kqi8/NRazO+hK3JS2j0b+NKGNCVhBB7JjjCoiZX/mqHlN5OJlq30DKTlwvb1TfXZpYmB1VCzxySz7YTJD4h3wq1Fk=
2021-12-05T01:22:51.856 INFO:teuthology.kill:2021-12-05 01:22:51,855.855 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-05T01:22:51.860 INFO:teuthology.kill:2021-12-05 01:22:51,860.860 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-05T01:22:51.861 INFO:teuthology.kill:2021-12-05 01:22:51,861.861 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-05T01:22:51.945 INFO:teuthology.kill:2021-12-05 01:22:51,945.945 INFO:teuthology.orchestra.console:Power off smithi143
2021-12-05T01:22:51.960 INFO:teuthology.kill:2021-12-05 01:22:51,960.960 INFO:teuthology.orchestra.console:Power off smithi134
2021-12-05T01:22:51.971 INFO:teuthology.kill:2021-12-05 01:22:51,971.971 INFO:teuthology.orchestra.console:Power off smithi195
2021-12-05T01:23:00.327 INFO:teuthology.kill:2021-12-05 01:23:00,327.327 INFO:teuthology.orchestra.console:Power off for smithi143 completed
2021-12-05T01:23:00.333 INFO:teuthology.kill:2021-12-05 01:23:00,333.333 INFO:teuthology.orchestra.console:Power off for smithi134 completed
2021-12-05T01:23:00.473 INFO:teuthology.kill:2021-12-05 01:23:00,472.472 INFO:teuthology.lock.ops:unlocked: smithi143.front.sepia.ceph.com
2021-12-05T01:23:00.480 INFO:teuthology.kill:2021-12-05 01:23:00,480.480 INFO:teuthology.lock.ops:unlocked: smithi134.front.sepia.ceph.com
2021-12-05T01:23:04.452 INFO:teuthology.kill:2021-12-05 01:23:04,451.451 INFO:teuthology.orchestra.console:Power off for smithi195 completed
2021-12-05T01:23:04.598 INFO:teuthology.kill:2021-12-05 01:23:04,598.598 INFO:teuthology.lock.ops:unlocked: smithi195.front.sepia.ceph.com
2021-12-05T01:25:04.840 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15