2022-07-02T08:05:50.809 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-07-02T08:05:50.955 INFO:teuthology.lock.ops:Start node 'smithi039.front.sepia.ceph.com' reimaging 2022-07-02T08:05:50.957 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: reset os type and version on server 2022-07-02T08:05:50.958 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2022-07-02T08:05:50.983 INFO:teuthology.lock.ops:Node 'smithi039.front.sepia.ceph.com' reimaging is complete 2022-07-02T08:05:50.983 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging 2022-07-02T08:05:50.983 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server 2022-07-02T08:05:50.984 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2022-07-02T08:05:51.009 INFO:teuthology.lock.ops:Node 'smithi050.front.sepia.ceph.com' reimaging is complete 2022-07-02T08:05:51.234 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of ubuntu 20.04 2022-07-02T08:05:51.312 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of ubuntu 20.04 2022-07-02T08:05:51.661 INFO:teuthology.orchestra.console:Power off smithi050 2022-07-02T08:05:51.675 INFO:teuthology.orchestra.console:Power off smithi039 2022-07-02T08:06:00.032 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2022-07-02T08:06:00.037 INFO:teuthology.orchestra.console:Power off for smithi039 completed 2022-07-02T08:06:00.134 INFO:teuthology.orchestra.console:Power on smithi050 2022-07-02T08:06:00.141 INFO:teuthology.orchestra.console:Power on smithi039 2022-07-02T08:06:08.508 INFO:teuthology.orchestra.console:Power on for smithi039 completed 2022-07-02T08:06:08.595 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2022-07-02T08:06:08.611 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish 2022-07-02T08:06:08.696 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2022-07-02T08:10:43.491 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-02T08:10:44.019 INFO:teuthology.provision.fog.smithi050:Node is ready 2022-07-02T08:10:44.026 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2022-07-02T08:10:44.032 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2022-07-02T08:10:44.202 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2022-07-02T08:11:50.118 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-02T08:11:50.609 INFO:teuthology.provision.fog.smithi039:Node is ready 2022-07-02T08:11:50.616 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039.front.sepia.ceph.com 2022-07-02T08:11:50.623 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39 smithi039.front.sepia.ceph.com smithi039 2022-07-02T08:11:50.773 INFO:teuthology.provision.fog.smithi039:Deploy complete! 2022-07-02T08:11:51.006 INFO:teuthology.lock.ops:Checking smithi039.front.sepia.ceph.com 2022-07-02T08:11:51.007 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-02T08:11:51.032 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2022-07-02T08:11:51.032 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-02T08:11:51.053 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: set os type and version on server 2022-07-02T08:11:51.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2022-07-02T08:11:51.286 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2022-07-02T08:11:51.292 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Ubuntu" 2022-07-02T08:11:51.293 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-02T08:11:51.293 INFO:teuthology.orchestra.run.smithi039.stdout:ID=ubuntu 2022-07-02T08:11:51.293 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE=debian 2022-07-02T08:11:51.293 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-02T08:11:51.293 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="20.04" 2022-07-02T08:11:51.294 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-02T08:11:51.294 INFO:teuthology.orchestra.run.smithi039.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-02T08:11:51.294 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-02T08:11:51.294 INFO:teuthology.orchestra.run.smithi039.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-02T08:11:51.295 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_CODENAME=focal 2022-07-02T08:11:51.295 INFO:teuthology.orchestra.run.smithi039.stdout:UBUNTU_CODENAME=focal 2022-07-02T08:11:51.296 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2022-07-02T08:11:51.316 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2022-07-02T08:11:51.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2022-07-02T08:11:51.672 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2022-07-02T08:11:51.679 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Ubuntu" 2022-07-02T08:11:51.680 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-02T08:11:51.680 INFO:teuthology.orchestra.run.smithi050.stdout:ID=ubuntu 2022-07-02T08:11:51.680 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE=debian 2022-07-02T08:11:51.680 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-02T08:11:51.681 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="20.04" 2022-07-02T08:11:51.681 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-02T08:11:51.681 INFO:teuthology.orchestra.run.smithi050.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-02T08:11:51.681 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-02T08:11:51.681 INFO:teuthology.orchestra.run.smithi050.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-02T08:11:51.682 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_CODENAME=focal 2022-07-02T08:11:51.682 INFO:teuthology.orchestra.run.smithi050.stdout:UBUNTU_CODENAME=focal 2022-07-02T08:11:51.682 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2022-07-02T08:11:51.791 INFO:teuthology.dispatcher.supervisor:Running job 6910290 2022-07-02T08:11:51.791 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/bin/teuthology -v --owner scheduled_nshelke@teuthology --archive /home/teuthworker/archive/nshelke-2022-07-02_04:41:48-fs:volumes-wip-nshelke-filter-internal-dirs-distro-default-smithi/6910290 --name nshelke-2022-07-02_04:41:48-fs:volumes-wip-nshelke-filter-internal-dirs-distro-default-smithi --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} -- /home/teuthworker/archive/nshelke-2022-07-02_04:41:48-fs:volumes-wip-nshelke-filter-internal-dirs-distro-default-smithi/6910290/orig.config.yaml 2022-07-02T08:11:51.795 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nshelke-2022-07-02_04:41:48-fs:volumes-wip-nshelke-filter-internal-dirs-distro-default-smithi/6910290 2022-07-02T08:11:51.795 INFO:teuthology.dispatcher.supervisor:Job PID: 4394 2022-07-02T08:11:51.796 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-02T20:12:49.607 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-02T20:12:49.783 INFO:teuthology.kill:Killing Pids: {4394} 2022-07-02T20:12:51.159 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-07-02T20:12:51.160 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-07-02T20:12:51.160 INFO:teuthology.misc:Compressing logs... 2022-07-02T20:12:51.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2022-07-02T20:12:51.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2022-07-02T20:12:52.894 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 290, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 348, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/remote.py", line 509, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi050 2022-07-02T20:12:52.966 INFO:teuthology.kill:No teuthology processes running 2022-07-02T20:12:54.290 INFO:teuthology.kill:Nuking machines: ['smithi039', 'smithi050'] 2022-07-02T20:12:55.453 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-02T20:12:55.454 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-02T20:12:55.454 INFO:teuthology.kill:2022-07-02 20:12:55,453.453 INFO:teuthology.nuke:targets: 2022-07-02T20:12:55.455 INFO:teuthology.kill: smithi039.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpaPpZL73GbwNbIXT5nvXZT6+DCY2L8nGYBoAxfHtMpBx3E3BMwZuY+3MVLePG9fDbtmkSFBlx8seFLfbVM4E8AE+TU8kNfueDH6YYJQPHVLTlzxEqqwe1qY+7e9Ax+fjjbT/1FYCRetvxq/HV5WosjLlYHKol9/TezjOIGtZ6QrflEudorOFEDW1oRrchumxtl6sig3SPD/pl//94bKSq8aRKHZX0wqNLhFXQkeM2K3mht1B13E6dUJ4YtyhOqR8ZbyffCRycJEEWjrp2xcQoWIOYtN01e/RqmgODV5NQVGWowZwH6mEi/mjzYy5GJ0G/VcZlPP7FYD87JCpsklJlaUrddhrm/tOPHuswCSV4sQK0xeBtrIWQhsznM8cPBj6vhIHBweFpsII0jPTlzxtPjaebgvuo6TOnlKr+AeCqDJ8rtNYjKSzaiv4T4PuBVshd/8+oUBILsNf6Nd3NryJmGGITqzyWPiPMMdBfkjDjZSd9Nzz06mzRveyNGx0A1Ys= 2022-07-02T20:12:55.455 INFO:teuthology.kill: smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm2REOzxnr2N2QK0eOAvw9ipmX4/Io0g33cSb4mjfUYCiOj0nMPoBfYEp5SkVFl/P7T/IdkdL5D0uIVNWUOSyROaGyJgN/XmHzpieu02M4n0wE7QvO0snXqsaonwTPGrhZ8QOVDF/ViqYj92Sn5FbWlTAKkNhVkfxEMu9nD3ZtJzix0N2Gaw0VsNygzyyEArcGUrmQIbRt2ApouNzpJeXxoXraCQFQ+c2+w8JJ8eHO+32SA6Mz6YjJwgC8xipt/wOyvmXBkojIvPSXXgepk7rTeOTRX3AucgJ2LUA+KPY7Msm+jsW86//c15QjvxPbAFBnTAxBXlRnmVmJnqp5v9SyJeNXEk8ZyC0tvkkluEIrPT+0Yqy50ZBBViarr3wlxlqeW3q5JzEWUgiUuXJU2na2JVIcULX7Ik4HwiAhtP925l4TNwlZLUEBuTbLkALqn8gK8nBm/b5+MoN/fHKb/s4d5ZgaOhGvGJ88Rp7RNzDXE0IbZ29qZGrKdjfnlIJFOB8= 2022-07-02T20:12:55.455 INFO:teuthology.kill:2022-07-02 20:12:55,453.453 INFO:teuthology.nuke:Checking targets against current locks 2022-07-02T20:12:55.507 INFO:teuthology.kill:2022-07-02 20:12:55,506.506 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-02T20:12:55.534 INFO:teuthology.kill:2022-07-02 20:12:55,534.534 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-02T20:12:55.590 INFO:teuthology.kill:2022-07-02 20:12:55,590.590 INFO:teuthology.orchestra.console:Power off smithi039 2022-07-02T20:12:55.632 INFO:teuthology.kill:2022-07-02 20:12:55,631.631 INFO:teuthology.orchestra.console:Power off smithi050 2022-07-02T20:13:03.968 INFO:teuthology.kill:2022-07-02 20:13:03,967.967 INFO:teuthology.orchestra.console:Power off for smithi039 completed 2022-07-02T20:13:04.116 INFO:teuthology.kill:2022-07-02 20:13:04,115.115 INFO:teuthology.lock.ops:unlocked: smithi039.front.sepia.ceph.com 2022-07-02T20:13:08.107 INFO:teuthology.kill:2022-07-02 20:13:08,106.106 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2022-07-02T20:13:08.256 INFO:teuthology.kill:2022-07-02 20:13:08,256.256 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com 2022-07-02T20:15:08.595 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-02T20:15:08.711 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi039 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-02_20:03:24-rgw-wip-yuri8-testing-2022-07-01-0822-pacific-distro-default-smithi/6910956 2022-07-02T20:15:08.711 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi050 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-02_20:03:24-rgw-wip-yuri8-testing-2022-07-01-0822-pacific-distro-default-smithi/6910956