2022-01-01T13:15:36.345 INFO:root:teuthology version: 1.1.0-95a7d479 2022-01-01T13:15:36.502 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi141.front.sepia.ceph.com' reimaging 2022-01-01T13:15:36.503 INFO:teuthology.lock.ops:Updating [ubuntu@smithi141.front.sepia.ceph.com]: reset os type and version on server 2022-01-01T13:15:36.503 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2022-01-01T13:15:36.521 INFO:teuthology.lock.ops:Node 'ubuntu@smithi141.front.sepia.ceph.com' reimaging is complete 2022-01-01T13:15:36.521 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi145.front.sepia.ceph.com' reimaging 2022-01-01T13:15:36.522 INFO:teuthology.lock.ops:Updating [ubuntu@smithi145.front.sepia.ceph.com]: reset os type and version on server 2022-01-01T13:15:36.522 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-01-01T13:15:36.540 INFO:teuthology.lock.ops:Node 'ubuntu@smithi145.front.sepia.ceph.com' reimaging is complete 2022-01-01T13:15:36.540 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging 2022-01-01T13:15:36.540 INFO:teuthology.lock.ops:Updating [ubuntu@smithi172.front.sepia.ceph.com]: reset os type and version on server 2022-01-01T13:15:36.541 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2022-01-01T13:15:36.560 INFO:teuthology.lock.ops:Node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging is complete 2022-01-01T13:15:36.710 INFO:teuthology.provision.fog.smithi141:Scheduling deploy of ubuntu 20.04 2022-01-01T13:15:36.744 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of ubuntu 20.04 2022-01-01T13:15:36.762 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of ubuntu 20.04 2022-01-01T13:15:37.322 INFO:teuthology.orchestra.console:Power off smithi145 2022-01-01T13:15:37.335 INFO:teuthology.orchestra.console:Power off smithi141 2022-01-01T13:15:37.419 INFO:teuthology.orchestra.console:Power off smithi172 2022-01-01T13:15:45.687 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-01-01T13:15:45.696 INFO:teuthology.orchestra.console:Power off for smithi141 completed 2022-01-01T13:15:45.781 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2022-01-01T13:15:45.788 INFO:teuthology.orchestra.console:Power on smithi145 2022-01-01T13:15:45.797 INFO:teuthology.orchestra.console:Power on smithi141 2022-01-01T13:15:45.882 INFO:teuthology.orchestra.console:Power on smithi172 2022-01-01T13:15:54.211 INFO:teuthology.orchestra.console:Power on for smithi141 completed 2022-01-01T13:15:54.211 INFO:teuthology.orchestra.console:Power on for smithi145 completed 2022-01-01T13:15:54.312 INFO:teuthology.provision.fog.smithi141:Waiting for deploy to finish 2022-01-01T13:15:54.314 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2022-01-01T13:15:54.349 INFO:teuthology.orchestra.console:Power on for smithi172 completed 2022-01-01T13:15:54.450 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2022-01-01T13:21:32.289 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-01T13:21:32.838 INFO:teuthology.provision.fog.smithi145:Node is ready 2022-01-01T13:21:32.845 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2022-01-01T13:21:32.892 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2022-01-01T13:21:33.111 INFO:teuthology.provision.fog.smithi145:Deploy complete! 2022-01-01T13:22:52.220 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-01T13:22:52.796 INFO:teuthology.provision.fog.smithi141:Node is ready 2022-01-01T13:22:52.803 INFO:teuthology.orchestra.run.smithi141.stdout:smithi141.front.sepia.ceph.com 2022-01-01T13:22:52.809 INFO:teuthology.orchestra.run.smithi141.stdout:172.21.15.141 smithi141.front.sepia.ceph.com smithi141 2022-01-01T13:22:52.983 INFO:teuthology.provision.fog.smithi141:Deploy complete! 2022-01-01T13:23:22.363 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-01T13:23:22.930 INFO:teuthology.provision.fog.smithi172:Node is ready 2022-01-01T13:23:22.936 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com 2022-01-01T13:23:22.942 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2022-01-01T13:23:23.126 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2022-01-01T13:23:23.336 INFO:teuthology.lock.ops:Checking smithi141.front.sepia.ceph.com 2022-01-01T13:23:23.337 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-01T13:23:23.356 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com 2022-01-01T13:23:23.357 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-01T13:23:23.377 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2022-01-01T13:23:23.377 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-01T13:23:23.395 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: set os type and version on server 2022-01-01T13:23:23.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2022-01-01T13:23:23.578 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64 2022-01-01T13:23:23.584 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="Ubuntu" 2022-01-01T13:23:23.585 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-01T13:23:23.585 INFO:teuthology.orchestra.run.smithi141.stdout:ID=ubuntu 2022-01-01T13:23:23.585 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE=debian 2022-01-01T13:23:23.585 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-01T13:23:23.585 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="20.04" 2022-01-01T13:23:23.586 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-01T13:23:23.586 INFO:teuthology.orchestra.run.smithi141.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-01T13:23:23.586 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-01T13:23:23.586 INFO:teuthology.orchestra.run.smithi141.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-01T13:23:23.587 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_CODENAME=focal 2022-01-01T13:23:23.587 INFO:teuthology.orchestra.run.smithi141.stdout:UBUNTU_CODENAME=focal 2022-01-01T13:23:23.588 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2022-01-01T13:23:23.607 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server 2022-01-01T13:23:23.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-01-01T13:23:23.791 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2022-01-01T13:23:23.797 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Ubuntu" 2022-01-01T13:23:23.797 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-01T13:23:23.797 INFO:teuthology.orchestra.run.smithi145.stdout:ID=ubuntu 2022-01-01T13:23:23.798 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE=debian 2022-01-01T13:23:23.798 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-01T13:23:23.798 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="20.04" 2022-01-01T13:23:23.798 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-01T13:23:23.799 INFO:teuthology.orchestra.run.smithi145.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-01T13:23:23.799 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-01T13:23:23.799 INFO:teuthology.orchestra.run.smithi145.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-01T13:23:23.799 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_CODENAME=focal 2022-01-01T13:23:23.799 INFO:teuthology.orchestra.run.smithi145.stdout:UBUNTU_CODENAME=focal 2022-01-01T13:23:23.800 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-01-01T13:23:23.819 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2022-01-01T13:23:23.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2022-01-01T13:23:23.985 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2022-01-01T13:23:24.033 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Ubuntu" 2022-01-01T13:23:24.034 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-01-01T13:23:24.034 INFO:teuthology.orchestra.run.smithi172.stdout:ID=ubuntu 2022-01-01T13:23:24.034 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE=debian 2022-01-01T13:23:24.034 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-01-01T13:23:24.035 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="20.04" 2022-01-01T13:23:24.035 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.ubuntu.com/" 2022-01-01T13:23:24.035 INFO:teuthology.orchestra.run.smithi172.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-01-01T13:23:24.035 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-01-01T13:23:24.035 INFO:teuthology.orchestra.run.smithi172.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-01-01T13:23:24.036 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_CODENAME=focal 2022-01-01T13:23:24.036 INFO:teuthology.orchestra.run.smithi172.stdout:UBUNTU_CODENAME=focal 2022-01-01T13:23:24.036 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2022-01-01T13:23:24.135 INFO:teuthology.dispatcher.supervisor:Running job 6589624 2022-01-01T13:23:24.136 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-01-01_13:15:01-multimds-octopus-distro-default-smithi/6589624 --name teuthology-2022-01-01_13:15:01-multimds-octopus-distro-default-smithi --description multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} -- /home/teuthworker/archive/teuthology-2022-01-01_13:15:01-multimds-octopus-distro-default-smithi/6589624/orig.config.yaml 2022-01-01T13:23:24.139 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-01-01_13:15:01-multimds-octopus-distro-default-smithi/6589624 2022-01-01T13:23:24.139 INFO:teuthology.dispatcher.supervisor:Job PID: 12950 2022-01-01T13:23:24.140 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-02T01:24:12.152 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-02T01:24:12.329 INFO:teuthology.kill:Killing Pids: {12950} 2022-01-02T01:24:13.443 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-01-02T01:24:13.444 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'mds.b', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-01-02T01:24:13.444 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['client.0', 'client.1'] 2022-01-02T01:24:13.445 INFO:teuthology.misc:Compressing logs... 2022-01-02T01:24:13.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2022-01-02T01:24:13.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-01-02T01:24:14.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2022-01-02T01:24:14.821 INFO:teuthology.kill:No teuthology processes running 2022-01-02T01:24:15.897 INFO:teuthology.kill:Nuking machines: ['smithi141', 'smithi145', 'smithi172'] 2022-01-02T01:24:16.832 INFO:teuthology.kill:2022-01-02 01:24:16,832.832 INFO:teuthology.nuke:targets: 2022-01-02T01:24:16.833 INFO:teuthology.kill: smithi141.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHzFaIL3x1C72rZsr5e4ybw5zNSaOgiKpVLCFrti6kqQUqbkOP0U5XQu0Q7XlByEm8MMF0azDWDaFGoCYdj2r0wEJ6vUwZLDVr6tMWdVDgf5jPblkQLvKc3G5fAhzu/6mrL20MSu7OiwquysUv43wq5uUH9ekLdEadd5xyiCXgkDt9z2XIsX2132ogl/eLNB9LyyKkynoUvfg/5XKAmHOzfL8C3drzvfkLUcCIsF+JS9IbzTFLk6TC07iU+6n7hyi2GSxlwTnr4FZz+WGEQ/u081G7TzR713TBJUkuOS05sNguQHOb6Z/PUcgxDHorSZwsve3Rv6vnyksX9YNQUd5FxqmUYVIDTNx/PHg4kKVKIkaPdAADUTrdfky8XTFDHoSfBm3jIjsv53fVwTc0z0ie0fBHngrljAOqrfXM2PQrN8PyWfXGRY35eeX2cbmyAOkSdcV0ln97IJyjGkFjW8tGxZRG8jKzE9awlTm5dzFKWaF/cidCq1aD+NrihVeUpxE= 2022-01-02T01:24:16.833 INFO:teuthology.kill: smithi145.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDK5woRcVJeOz/LBRmpCmAgf2Js9RXoopyJNI03gh+1jZFecFGU4pcI/1r+Hhdwfa9n9KZZYWl4f9EqG1Gv7uaRcbufJlc6izHVxVm6YqJqtAI71EMFvZdezvlLQb94Yg0s1KifB77E3ATkKF4qJKozfwdwF1qSCELTbm7knuniXuZMUcXwAux2ZckmjSO6uDF9E9z/RPzMVWHut7mdneScA5YKhUmbvSNBhUlgBJ8yN+4oufRmwqidoMX+H4D6jkmkzvP3aiRoUTarq5V/2z2JKa70+wR6/YIlJ0GJIoWYLlgXiaPNY1x8oSfPCqe14k+ED3qNgG7qflICcR96SVjaPpUL+EAoizmbIQJbyFqqECgojZRe8Tu7ZG/6Mpaq9TAKeSvQMhqUrHXOUq8WeJ/doc6z9di52+bUyJmY2488UFkPFqrH+xV2cYVXsS1RPIuSWpPbHZ6YZGA7jef9sIY+2y3xr7QgTMzEhXi3yzdTPsHYzVE5Ncwr4gNq9j45LHs= 2022-01-02T01:24:16.834 INFO:teuthology.kill: smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8umXULdAEsE3o/E3BKg7ipCXJ47BpFY6kJlDRFvHmXi7ymrcM2FAd+g+hYdgGPeZXFljlohiCKoaoeCOA9gW/YHDNK3EObfv5olNFlAJjmzYXhxini0nvy3VwLAAhh7rVlWy/REIyqe/rQ+f024/O/0g39XSPZbnmLHqn8Vr7WP2BgaMD3v5mP5921r08yuzpcSKiXau7VZ3ynhmiSAasJcZ9USa3HB3qE9lOLCbzAbzaD9i1UxnPVP0wqbzFb55Ac/wBzP6zlP29s4a8VBSKxRsZw7/lNkNhsDWKyZ7y6mrkvd+Vaao4ndfE1VpPKFnXS1yWuA4WoUuzOpPAM48Z6PVwooEFMwBxxPMEFJjxq0BIyZylUWA0pPceSq8foakFmn+urxOLIWgmpXXBvOWEI7LrahRjSiqNgg3c+8hgs41SywauT7iHfHcVQ0+lUuM6BKzlYunaHQKtYE0189woliGQ2zvWZkgo5CEPb55Z/0WsTkIPdwS//CxbQTfRpE8= 2022-01-02T01:24:16.860 INFO:teuthology.kill:2022-01-02 01:24:16,860.860 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-02T01:24:16.864 INFO:teuthology.kill:2022-01-02 01:24:16,864.864 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-02T01:24:16.865 INFO:teuthology.kill:2022-01-02 01:24:16,865.865 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-02T01:24:16.945 INFO:teuthology.kill:2022-01-02 01:24:16,945.945 INFO:teuthology.orchestra.console:Power off smithi145 2022-01-02T01:24:16.960 INFO:teuthology.kill:2022-01-02 01:24:16,959.959 INFO:teuthology.orchestra.console:Power off smithi141 2022-01-02T01:24:16.971 INFO:teuthology.kill:2022-01-02 01:24:16,971.971 INFO:teuthology.orchestra.console:Power off smithi172 2022-01-02T01:24:29.441 INFO:teuthology.kill:2022-01-02 01:24:29,441.441 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-01-02T01:24:29.446 INFO:teuthology.kill:2022-01-02 01:24:29,446.446 INFO:teuthology.orchestra.console:Power off for smithi141 completed 2022-01-02T01:24:29.451 INFO:teuthology.kill:2022-01-02 01:24:29,451.451 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2022-01-02T01:24:29.597 INFO:teuthology.kill:2022-01-02 01:24:29,596.596 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com 2022-01-02T01:24:29.602 INFO:teuthology.kill:2022-01-02 01:24:29,602.602 INFO:teuthology.lock.ops:unlocked: smithi141.front.sepia.ceph.com 2022-01-02T01:24:29.603 INFO:teuthology.kill:2022-01-02 01:24:29,603.603 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com 2022-01-02T01:26:29.954 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-02T01:26:30.078 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi141 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-02_01:25:02-upgrade:pacific-p2p-pacific-distro-default-smithi/6589730 2022-01-02T01:26:30.079 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi145 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-02_01:25:02-upgrade:pacific-p2p-pacific-distro-default-smithi/6589730 2022-01-02T01:26:30.079 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi172 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-02_01:25:02-upgrade:pacific-p2p-pacific-distro-default-smithi/6589730