2022-03-24T09:45:46.882 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-24T09:45:47.017 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging 2022-03-24T09:45:47.019 INFO:teuthology.lock.ops:Updating [ubuntu@smithi176.front.sepia.ceph.com]: reset os type and version on server 2022-03-24T09:45:47.020 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2022-03-24T09:45:47.040 INFO:teuthology.lock.ops:Node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging is complete 2022-03-24T09:45:47.040 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi201.front.sepia.ceph.com' reimaging 2022-03-24T09:45:47.040 INFO:teuthology.lock.ops:Updating [ubuntu@smithi201.front.sepia.ceph.com]: reset os type and version on server 2022-03-24T09:45:47.041 INFO:teuthology.lock.ops:Updating smithi201.front.sepia.ceph.com on lock server 2022-03-24T09:45:47.059 INFO:teuthology.lock.ops:Node 'ubuntu@smithi201.front.sepia.ceph.com' reimaging is complete 2022-03-24T09:45:47.243 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of ubuntu 20.04 2022-03-24T09:45:47.256 INFO:teuthology.provision.fog.smithi201:Scheduling deploy of ubuntu 20.04 2022-03-24T09:45:47.537 INFO:teuthology.orchestra.console:Power off smithi176 2022-03-24T09:45:47.555 INFO:teuthology.orchestra.console:Power off smithi201 2022-03-24T09:45:47.700 INFO:teuthology.orchestra.console:Power off for smithi201 completed 2022-03-24T09:45:47.709 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2022-03-24T09:45:47.802 INFO:teuthology.orchestra.console:Power on smithi201 2022-03-24T09:45:47.810 INFO:teuthology.orchestra.console:Power on smithi176 2022-03-24T09:45:56.173 INFO:teuthology.orchestra.console:Power on for smithi201 completed 2022-03-24T09:45:56.180 INFO:teuthology.orchestra.console:Power on for smithi176 completed 2022-03-24T09:45:56.275 INFO:teuthology.provision.fog.smithi201:Waiting for deploy to finish 2022-03-24T09:45:56.280 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish 2022-03-24T09:50:47.924 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-24T09:50:48.387 INFO:teuthology.provision.fog.smithi201:Node is ready 2022-03-24T09:50:48.393 INFO:teuthology.orchestra.run.smithi201.stdout:smithi201.front.sepia.ceph.com 2022-03-24T09:50:48.441 INFO:teuthology.orchestra.run.smithi201.stdout:172.21.15.201 smithi201.front.sepia.ceph.com smithi201 2022-03-24T09:50:48.587 INFO:teuthology.provision.fog.smithi201:Deploy complete! 2022-03-24T09:50:54.175 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-24T09:50:54.585 INFO:teuthology.provision.fog.smithi176:Node is ready 2022-03-24T09:50:54.592 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com 2022-03-24T09:50:54.599 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176 2022-03-24T09:50:54.809 INFO:teuthology.provision.fog.smithi176:Deploy complete! 2022-03-24T09:50:54.987 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com 2022-03-24T09:50:54.988 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-24T09:50:55.006 INFO:teuthology.lock.ops:Checking smithi201.front.sepia.ceph.com 2022-03-24T09:50:55.007 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-24T09:50:55.024 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server 2022-03-24T09:50:55.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2022-03-24T09:50:55.206 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2022-03-24T09:50:55.213 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Ubuntu" 2022-03-24T09:50:55.213 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-24T09:50:55.214 INFO:teuthology.orchestra.run.smithi176.stdout:ID=ubuntu 2022-03-24T09:50:55.214 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE=debian 2022-03-24T09:50:55.214 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-24T09:50:55.215 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="20.04" 2022-03-24T09:50:55.215 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-24T09:50:55.215 INFO:teuthology.orchestra.run.smithi176.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-24T09:50:55.215 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-24T09:50:55.215 INFO:teuthology.orchestra.run.smithi176.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-24T09:50:55.216 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_CODENAME=focal 2022-03-24T09:50:55.216 INFO:teuthology.orchestra.run.smithi176.stdout:UBUNTU_CODENAME=focal 2022-03-24T09:50:55.217 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2022-03-24T09:50:55.234 INFO:teuthology.lock.ops:Updating [smithi201.front.sepia.ceph.com]: set os type and version on server 2022-03-24T09:50:55.235 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi201.front.sepia.ceph.com' 2022-03-24T09:50:55.411 INFO:teuthology.orchestra.run.smithi201.stdout:x86_64 2022-03-24T09:50:55.418 INFO:teuthology.orchestra.run.smithi201.stdout:NAME="Ubuntu" 2022-03-24T09:50:55.418 INFO:teuthology.orchestra.run.smithi201.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-24T09:50:55.419 INFO:teuthology.orchestra.run.smithi201.stdout:ID=ubuntu 2022-03-24T09:50:55.419 INFO:teuthology.orchestra.run.smithi201.stdout:ID_LIKE=debian 2022-03-24T09:50:55.419 INFO:teuthology.orchestra.run.smithi201.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-24T09:50:55.419 INFO:teuthology.orchestra.run.smithi201.stdout:VERSION_ID="20.04" 2022-03-24T09:50:55.420 INFO:teuthology.orchestra.run.smithi201.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-24T09:50:55.420 INFO:teuthology.orchestra.run.smithi201.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-24T09:50:55.420 INFO:teuthology.orchestra.run.smithi201.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-24T09:50:55.420 INFO:teuthology.orchestra.run.smithi201.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-24T09:50:55.421 INFO:teuthology.orchestra.run.smithi201.stdout:VERSION_CODENAME=focal 2022-03-24T09:50:55.421 INFO:teuthology.orchestra.run.smithi201.stdout:UBUNTU_CODENAME=focal 2022-03-24T09:50:55.421 INFO:teuthology.lock.ops:Updating smithi201.front.sepia.ceph.com on lock server 2022-03-24T09:50:55.515 INFO:teuthology.dispatcher.supervisor:Running job 6755791 2022-03-24T09:50:55.516 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/6755791 --name yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi --description fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/valgrind} -- /home/teuthworker/archive/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/6755791/orig.config.yaml 2022-03-24T09:50:55.519 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/6755791 2022-03-24T09:50:55.520 INFO:teuthology.dispatcher.supervisor:Job PID: 15714 2022-03-24T09:50:55.520 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-24T16:21:25.495 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-24T16:21:25.669 INFO:teuthology.kill:Killing Pids: {15714} 2022-03-24T16:21:26.833 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-24T16:21:26.834 INFO:teuthology.task.internal:roles: ubuntu@smithi201.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-24T16:21:26.834 INFO:teuthology.misc:Compressing logs... 2022-03-24T16:21:26.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2022-03-24T16:22:26.899 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/teuthology/dispatcher/supervisor.py", line 287, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/teuthology/dispatcher/supervisor.py", line 345, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2022-03-24T16:22:26.994 INFO:teuthology.kill:No teuthology processes running 2022-03-24T16:22:28.129 INFO:teuthology.kill:Nuking machines: ['smithi176', 'smithi201'] 2022-03-24T16:22:29.109 INFO:teuthology.kill:2022-03-24 16:22:29,108.108 INFO:teuthology.nuke:targets: 2022-03-24T16:22:29.109 INFO:teuthology.kill: smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY3kDlqxv2hL2dSrULkbu7OXpCSK3imEXwM6V3qD2ddJgSUXcrErZrk0ypp2vDxajSOf+W623JLOPh7mydRPyb4x9/fM1ds+ltK9cbWa90W3Hsu3vO5TInYxB6GH+8x/YTEWVQ10COB2Rhak1aKygu4Ands1Vloc6n0a+GjPDcS97JrMgAYoLcYhMiNEaBNcKykghplFWi8IeO3PRRnz5QSNlcQ6rO2OIBXJXay5kagCZm4sAvSqcQ2jtV2hmgfNbbzjTVosnI9hu5E6+4r6sRDviBUGR1cNTW4VIq8Pm7vSGbiv+CsmrXYgTXf1UCBu6QGFs5jos43+nE4qOIbEos6x7cEETGB35ZqEP7TY9DYOtucNTvxcAa02HJtPEVZHgkbJKxXOCK+ETilcJ4cdQzkClLMxR+nCl+Hyai/6H662b+R1soGLEsmHa1yydGEF05xq9S4It4LXYdQ4KhImNL0QWwszee8VyYEK8f9HP3pZkw6bmf6Y5/shv0aUD14SE= 2022-03-24T16:22:29.110 INFO:teuthology.kill: smithi201.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE8QkYA1t74hTdpj9kETi3O4XrI7E94+709DZx9fcux1+Mg1CU2rJkAuMcweiIVHaO7tFF2umjXoWDp3KoxTKWuVuLSz+DRvRyI180aeeSu9uP4bgIilfX8iYCv9/kzFgQTx+gyFoxlrCqaQpKCdm1DZgHKfvhxntEa+PlIsCIp5qcPdfNhRNeyNuggPej/2I4P8y9GfxhRd4m67onKY025cpvZgcFk0nJ3su3mhDus2AcIRrNGldHkTgkvBn3J4XJ/thKQdAxANbsfsdK5778lJiQ7bGYgmsbgwvwUgnoXts/hhELShnG1ukVkK30E0tTuRlcIkpcOrmF+uaKsDgRnDmocFZb0ZUYcDwPVGygli1raoTBWak2NvuFo8LmJszHG3AukoZKUrLmIclGarBbzjikr+ctdmwGjGccz/EwVsHgbRUmR9u4lDz0Gp6lv08Nq5iqO3dmbQwR/MhXdlv6jtHfynH+uPP8UXHtXUWUB88UAX9e0Mt2b2Rw2kqe8zM= 2022-03-24T16:22:29.136 INFO:teuthology.kill:2022-03-24 16:22:29,136.136 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-24T16:22:29.141 INFO:teuthology.kill:2022-03-24 16:22:29,141.141 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-24T16:22:29.210 INFO:teuthology.kill:2022-03-24 16:22:29,210.210 INFO:teuthology.orchestra.console:Power off smithi176 2022-03-24T16:22:29.226 INFO:teuthology.kill:2022-03-24 16:22:29,226.226 INFO:teuthology.orchestra.console:Power off smithi201 2022-03-24T16:22:41.693 INFO:teuthology.kill:2022-03-24 16:22:41,692.692 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2022-03-24T16:22:41.698 INFO:teuthology.kill:2022-03-24 16:22:41,698.698 INFO:teuthology.orchestra.console:Power off for smithi201 completed 2022-03-24T16:22:41.843 INFO:teuthology.kill:2022-03-24 16:22:41,842.842 INFO:teuthology.lock.ops:unlocked: smithi201.front.sepia.ceph.com 2022-03-24T16:22:41.851 INFO:teuthology.kill:2022-03-24 16:22:41,851.851 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com 2022-03-24T16:24:42.200 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-24T16:24:42.316 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi176 but it was locked by another job: /home/teuthworker/archive/kchai-2022-03-23_11:17:34-rados-wip-cxx20-fixes-mon-1-distro-default-smithi/6755384 2022-03-24T16:24:42.316 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi201 but it was locked by another job: /home/teuthworker/archive/kchai-2022-03-23_11:17:34-rados-wip-cxx20-fixes-mon-1-distro-default-smithi/6755385