2021-10-04T23:21:55.213 INFO:root:teuthology version: 1.1.0-f08b273e
2021-10-04T23:21:55.361 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi124.front.sepia.ceph.com' reimaging
2021-10-04T23:21:55.363 INFO:teuthology.lock.ops:Updating [ubuntu@smithi124.front.sepia.ceph.com]: reset os type and version on server
2021-10-04T23:21:55.363 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server
2021-10-04T23:21:55.384 INFO:teuthology.lock.ops:Node 'ubuntu@smithi124.front.sepia.ceph.com' reimaging is complete
2021-10-04T23:21:55.384 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi125.front.sepia.ceph.com' reimaging
2021-10-04T23:21:55.385 INFO:teuthology.lock.ops:Updating [ubuntu@smithi125.front.sepia.ceph.com]: reset os type and version on server
2021-10-04T23:21:55.385 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2021-10-04T23:21:55.406 INFO:teuthology.lock.ops:Node 'ubuntu@smithi125.front.sepia.ceph.com' reimaging is complete
2021-10-04T23:21:55.406 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging
2021-10-04T23:21:55.407 INFO:teuthology.lock.ops:Updating [ubuntu@smithi155.front.sepia.ceph.com]: reset os type and version on server
2021-10-04T23:21:55.407 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server
2021-10-04T23:21:55.427 INFO:teuthology.lock.ops:Node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging is complete
2021-10-04T23:21:55.603 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of ubuntu 20.04
2021-10-04T23:21:55.637 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of ubuntu 20.04
2021-10-04T23:21:55.642 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of ubuntu 20.04
2021-10-04T23:21:56.116 INFO:teuthology.orchestra.console:Power off smithi155
2021-10-04T23:21:56.135 INFO:teuthology.orchestra.console:Power off smithi125
2021-10-04T23:21:56.172 INFO:teuthology.orchestra.console:Power off smithi124
2021-10-04T23:21:56.284 INFO:teuthology.orchestra.console:Power off for smithi155 completed
2021-10-04T23:21:56.306 INFO:teuthology.orchestra.console:Power off for smithi125 completed
2021-10-04T23:21:56.317 INFO:teuthology.orchestra.console:Power off for smithi124 completed
2021-10-04T23:21:56.386 INFO:teuthology.orchestra.console:Power on smithi155
2021-10-04T23:21:56.415 INFO:teuthology.orchestra.console:Power on smithi125
2021-10-04T23:21:56.422 INFO:teuthology.orchestra.console:Power on smithi124
2021-10-04T23:22:00.674 INFO:teuthology.orchestra.console:Power on for smithi124 completed
2021-10-04T23:22:00.775 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish
2021-10-04T23:22:04.758 INFO:teuthology.orchestra.console:Power on for smithi155 completed
2021-10-04T23:22:04.776 INFO:teuthology.orchestra.console:Power on for smithi125 completed
2021-10-04T23:22:04.861 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish
2021-10-04T23:22:04.878 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish
2021-10-04T23:26:06.825 ERROR:teuthology.orchestra.connection:Error authenticating with smithi125.front.sepia.ceph.com: Authentication timeout.
2021-10-04T23:26:56.459 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2021-10-04T23:27:13.737 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-04T23:27:14.223 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-04T23:27:14.283 INFO:teuthology.provision.fog.smithi124:Node is ready
2021-10-04T23:27:14.332 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com
2021-10-04T23:27:14.342 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124
2021-10-04T23:27:14.556 INFO:teuthology.provision.fog.smithi124:Deploy complete!
2021-10-04T23:27:14.748 INFO:teuthology.provision.fog.smithi125:Node is ready
2021-10-04T23:27:14.756 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com
2021-10-04T23:27:14.809 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125
2021-10-04T23:27:14.967 INFO:teuthology.provision.fog.smithi125:Deploy complete!
2021-10-04T23:27:27.039 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-04T23:27:27.606 INFO:teuthology.provision.fog.smithi155:Node is ready
2021-10-04T23:27:27.613 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com
2021-10-04T23:27:27.665 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155
2021-10-04T23:27:27.868 INFO:teuthology.provision.fog.smithi155:Deploy complete!
2021-10-04T23:27:28.110 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com
2021-10-04T23:27:28.111 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-04T23:27:28.130 INFO:teuthology.lock.ops:Checking smithi125.front.sepia.ceph.com
2021-10-04T23:27:28.131 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-04T23:27:28.149 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com
2021-10-04T23:27:28.150 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-04T23:27:28.166 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server
2021-10-04T23:27:28.167 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com'
2021-10-04T23:27:28.479 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64
2021-10-04T23:27:28.486 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Ubuntu"
2021-10-04T23:27:28.486 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-10-04T23:27:28.486 INFO:teuthology.orchestra.run.smithi124.stdout:ID=ubuntu
2021-10-04T23:27:28.487 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE=debian
2021-10-04T23:27:28.487 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-10-04T23:27:28.487 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="20.04"
2021-10-04T23:27:28.488 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.ubuntu.com/"
2021-10-04T23:27:28.488 INFO:teuthology.orchestra.run.smithi124.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-10-04T23:27:28.488 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-10-04T23:27:28.488 INFO:teuthology.orchestra.run.smithi124.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-10-04T23:27:28.489 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_CODENAME=focal
2021-10-04T23:27:28.489 INFO:teuthology.orchestra.run.smithi124.stdout:UBUNTU_CODENAME=focal
2021-10-04T23:27:28.490 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server
2021-10-04T23:27:28.508 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: set os type and version on server
2021-10-04T23:27:28.509 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com'
2021-10-04T23:27:28.737 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64
2021-10-04T23:27:28.744 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="Ubuntu"
2021-10-04T23:27:28.745 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-10-04T23:27:28.745 INFO:teuthology.orchestra.run.smithi125.stdout:ID=ubuntu
2021-10-04T23:27:28.745 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE=debian
2021-10-04T23:27:28.745 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-10-04T23:27:28.746 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="20.04"
2021-10-04T23:27:28.746 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://www.ubuntu.com/"
2021-10-04T23:27:28.746 INFO:teuthology.orchestra.run.smithi125.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-10-04T23:27:28.746 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-10-04T23:27:28.747 INFO:teuthology.orchestra.run.smithi125.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-10-04T23:27:28.747 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_CODENAME=focal
2021-10-04T23:27:28.747 INFO:teuthology.orchestra.run.smithi125.stdout:UBUNTU_CODENAME=focal
2021-10-04T23:27:28.748 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2021-10-04T23:27:28.770 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server
2021-10-04T23:27:28.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com'
2021-10-04T23:27:28.942 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64
2021-10-04T23:27:28.948 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Ubuntu"
2021-10-04T23:27:28.949 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-10-04T23:27:28.949 INFO:teuthology.orchestra.run.smithi155.stdout:ID=ubuntu
2021-10-04T23:27:28.949 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE=debian
2021-10-04T23:27:28.949 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-10-04T23:27:28.950 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="20.04"
2021-10-04T23:27:28.950 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.ubuntu.com/"
2021-10-04T23:27:28.950 INFO:teuthology.orchestra.run.smithi155.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-10-04T23:27:28.951 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-10-04T23:27:28.951 INFO:teuthology.orchestra.run.smithi155.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-10-04T23:27:28.951 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_CODENAME=focal
2021-10-04T23:27:28.951 INFO:teuthology.orchestra.run.smithi155.stdout:UBUNTU_CODENAME=focal
2021-10-04T23:27:28.952 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server
2021-10-04T23:27:29.041 INFO:teuthology.dispatcher.supervisor:Running job 6422173
2021-10-04T23:27:29.042 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-10-04_22:44:34-fs-wip-pdonnell-testing-20211002.163337-distro-basic-smithi/6422173 --name pdonnell-2021-10-04_22:44:34-fs-wip-pdonnell-testing-20211002.163337-distro-basic-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} -- /home/teuthworker/archive/pdonnell-2021-10-04_22:44:34-fs-wip-pdonnell-testing-20211002.163337-distro-basic-smithi/6422173/orig.config.yaml
2021-10-04T23:27:29.045 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-10-04_22:44:34-fs-wip-pdonnell-testing-20211002.163337-distro-basic-smithi/6422173
2021-10-04T23:27:29.045 INFO:teuthology.dispatcher.supervisor:Job PID: 12669
2021-10-04T23:27:29.046 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-10-04T23:33:29.235 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-10-04T23:33:29.380 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-10-04T23:33:29.381 INFO:teuthology.nuke:Checking targets against current locks
2021-10-04T23:33:29.509 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-04T23:33:29.514 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-04T23:33:29.516 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-04T23:33:29.598 INFO:teuthology.orchestra.console:Power off smithi155
2021-10-04T23:33:29.616 INFO:teuthology.orchestra.console:Power off smithi125
2021-10-04T23:33:29.629 INFO:teuthology.orchestra.console:Power off smithi124
2021-10-04T23:33:37.985 INFO:teuthology.orchestra.console:Power off for smithi125 completed
2021-10-04T23:33:37.995 INFO:teuthology.orchestra.console:Power off for smithi124 completed
2021-10-04T23:33:38.126 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com
2021-10-04T23:33:42.100 INFO:teuthology.orchestra.console:Power off for smithi155 completed
2021-10-04T23:34:05.289 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/virtualenv/bin/teuthology-dispatcher", line 33, in <module>
    sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')())
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/scripts/dispatcher.py", line 33, in main
    teuthology.dispatcher.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/dispatcher/__init__.py", line 61, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/dispatcher/supervisor.py", line 59, in main
    verbose
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/dispatcher/supervisor.py", line 152, in run_job
    unlock_targets(job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/dispatcher/supervisor.py", line 196, in unlock_targets
    nuke(fake_ctx, True)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/nuke/__init__.py", line 268, in nuke
    for unnuked in p:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/nuke/__init__.py", line 299, in nuke_one
    unlock_one(ctx, list(target.keys())[0], ctx.owner)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/lock/ops.py", line 204, in unlock_one
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'unlock smithi124.front.sepia.ceph.com' reached maximum tries (10) after waiting for 32.5 seconds