2022-02-01T13:11:25.601 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-02-01T13:11:25.761 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging 2022-02-01T13:11:25.762 INFO:teuthology.lock.ops:Updating [ubuntu@smithi068.front.sepia.ceph.com]: reset os type and version on server 2022-02-01T13:11:25.763 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server 2022-02-01T13:11:25.783 INFO:teuthology.lock.ops:Node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging is complete 2022-02-01T13:11:25.783 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging 2022-02-01T13:11:25.784 INFO:teuthology.lock.ops:Updating [ubuntu@smithi106.front.sepia.ceph.com]: reset os type and version on server 2022-02-01T13:11:25.784 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2022-02-01T13:11:25.803 INFO:teuthology.lock.ops:Node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging is complete 2022-02-01T13:11:25.803 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging 2022-02-01T13:11:25.804 INFO:teuthology.lock.ops:Updating [ubuntu@smithi112.front.sepia.ceph.com]: reset os type and version on server 2022-02-01T13:11:25.804 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2022-02-01T13:11:25.823 INFO:teuthology.lock.ops:Node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging is complete 2022-02-01T13:11:26.003 INFO:teuthology.provision.fog.smithi068:Scheduling deploy of ubuntu 20.04 2022-02-01T13:11:26.022 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of ubuntu 20.04 2022-02-01T13:11:26.046 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of ubuntu 20.04 2022-02-01T13:11:26.340 INFO:teuthology.orchestra.console:Power off smithi068 2022-02-01T13:11:26.355 INFO:teuthology.orchestra.console:Power off smithi106 2022-02-01T13:11:26.401 INFO:teuthology.orchestra.console:Power off smithi112 2022-02-01T13:11:26.509 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2022-02-01T13:11:26.610 INFO:teuthology.orchestra.console:Power on smithi106 2022-02-01T13:11:34.971 INFO:teuthology.orchestra.console:Power on for smithi106 completed 2022-02-01T13:11:35.073 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish 2022-02-01T13:11:38.829 INFO:teuthology.orchestra.console:Power off for smithi068 completed 2022-02-01T13:11:38.873 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2022-02-01T13:11:38.931 INFO:teuthology.orchestra.console:Power on smithi068 2022-02-01T13:11:38.975 INFO:teuthology.orchestra.console:Power on smithi112 2022-02-01T13:11:43.181 INFO:teuthology.orchestra.console:Power on for smithi068 completed 2022-02-01T13:11:43.224 INFO:teuthology.orchestra.console:Power on for smithi112 completed 2022-02-01T13:11:43.282 INFO:teuthology.provision.fog.smithi068:Waiting for deploy to finish 2022-02-01T13:11:43.325 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2022-02-01T13:16:45.267 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-01T13:16:45.853 INFO:teuthology.provision.fog.smithi112:Node is ready 2022-02-01T13:16:45.860 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2022-02-01T13:16:45.868 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2022-02-01T13:16:46.059 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2022-02-01T13:16:50.804 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-01T13:16:51.355 INFO:teuthology.provision.fog.smithi106:Node is ready 2022-02-01T13:16:51.405 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com 2022-02-01T13:16:51.413 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106 2022-02-01T13:16:51.603 INFO:teuthology.provision.fog.smithi106:Deploy complete! 2022-02-01T13:16:58.891 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-01T13:16:59.431 INFO:teuthology.provision.fog.smithi068:Node is ready 2022-02-01T13:16:59.438 INFO:teuthology.orchestra.run.smithi068.stdout:smithi068.front.sepia.ceph.com 2022-02-01T13:16:59.445 INFO:teuthology.orchestra.run.smithi068.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068 2022-02-01T13:16:59.644 INFO:teuthology.provision.fog.smithi068:Deploy complete! 2022-02-01T13:16:59.884 INFO:teuthology.lock.ops:Checking smithi068.front.sepia.ceph.com 2022-02-01T13:16:59.885 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-01T13:16:59.910 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com 2022-02-01T13:16:59.910 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-01T13:16:59.925 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2022-02-01T13:16:59.925 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-01T13:16:59.941 INFO:teuthology.lock.ops:Updating [smithi068.front.sepia.ceph.com]: set os type and version on server 2022-02-01T13:16:59.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com' 2022-02-01T13:17:00.110 INFO:teuthology.orchestra.run.smithi068.stdout:x86_64 2022-02-01T13:17:00.117 INFO:teuthology.orchestra.run.smithi068.stdout:NAME="Ubuntu" 2022-02-01T13:17:00.117 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-01T13:17:00.117 INFO:teuthology.orchestra.run.smithi068.stdout:ID=ubuntu 2022-02-01T13:17:00.118 INFO:teuthology.orchestra.run.smithi068.stdout:ID_LIKE=debian 2022-02-01T13:17:00.118 INFO:teuthology.orchestra.run.smithi068.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-01T13:17:00.118 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_ID="20.04" 2022-02-01T13:17:00.118 INFO:teuthology.orchestra.run.smithi068.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-01T13:17:00.118 INFO:teuthology.orchestra.run.smithi068.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-01T13:17:00.119 INFO:teuthology.orchestra.run.smithi068.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-01T13:17:00.119 INFO:teuthology.orchestra.run.smithi068.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-01T13:17:00.119 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_CODENAME=focal 2022-02-01T13:17:00.119 INFO:teuthology.orchestra.run.smithi068.stdout:UBUNTU_CODENAME=focal 2022-02-01T13:17:00.120 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server 2022-02-01T13:17:00.136 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server 2022-02-01T13:17:00.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2022-02-01T13:17:00.314 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2022-02-01T13:17:00.361 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Ubuntu" 2022-02-01T13:17:00.361 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-01T13:17:00.362 INFO:teuthology.orchestra.run.smithi106.stdout:ID=ubuntu 2022-02-01T13:17:00.362 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE=debian 2022-02-01T13:17:00.362 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-01T13:17:00.363 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="20.04" 2022-02-01T13:17:00.363 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-01T13:17:00.363 INFO:teuthology.orchestra.run.smithi106.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-01T13:17:00.363 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-01T13:17:00.363 INFO:teuthology.orchestra.run.smithi106.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-01T13:17:00.364 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_CODENAME=focal 2022-02-01T13:17:00.364 INFO:teuthology.orchestra.run.smithi106.stdout:UBUNTU_CODENAME=focal 2022-02-01T13:17:00.364 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2022-02-01T13:17:00.380 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2022-02-01T13:17:00.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2022-02-01T13:17:00.646 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2022-02-01T13:17:00.693 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Ubuntu" 2022-02-01T13:17:00.694 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-01T13:17:00.694 INFO:teuthology.orchestra.run.smithi112.stdout:ID=ubuntu 2022-02-01T13:17:00.694 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE=debian 2022-02-01T13:17:00.694 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-01T13:17:00.695 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="20.04" 2022-02-01T13:17:00.695 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-01T13:17:00.695 INFO:teuthology.orchestra.run.smithi112.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-01T13:17:00.695 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-01T13:17:00.695 INFO:teuthology.orchestra.run.smithi112.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-01T13:17:00.696 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_CODENAME=focal 2022-02-01T13:17:00.696 INFO:teuthology.orchestra.run.smithi112.stdout:UBUNTU_CODENAME=focal 2022-02-01T13:17:00.696 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2022-02-01T13:17:00.795 INFO:teuthology.dispatcher.supervisor:Running job 6652153 2022-02-01T13:17:00.795 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-02-01_03:15:04-fs-master-distro-default-smithi/6652153 --name teuthology-2022-02-01_03:15:04-fs-master-distro-default-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} -- /home/teuthworker/archive/teuthology-2022-02-01_03:15:04-fs-master-distro-default-smithi/6652153/orig.config.yaml 2022-02-01T13:17:00.798 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-02-01_03:15:04-fs-master-distro-default-smithi/6652153 2022-02-01T13:17:00.799 INFO:teuthology.dispatcher.supervisor:Job PID: 5212 2022-02-01T13:17:00.799 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-01T19:47:29.308 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-01T19:47:29.503 INFO:teuthology.kill:Killing Pids: {5212} 2022-02-01T19:47:30.568 INFO:teuthology.task.internal:roles: ubuntu@smithi068.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2022-02-01T19:47:30.569 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2022-02-01T19:47:30.569 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2022-02-01T19:47:30.570 INFO:teuthology.misc:Compressing logs... 2022-02-01T19:47:30.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com' 2022-02-01T19:47:30.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2022-02-01T19:47:31.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2022-02-01T19:47:31.393 INFO:teuthology.misc:Compressing logs... 2022-02-01T19:47:31.582 INFO:teuthology.misc:Compressing logs... 2022-02-01T19:49:16.771 INFO:teuthology.kill:No teuthology processes running 2022-02-01T19:49:17.933 INFO:teuthology.kill:Nuking machines: ['smithi068', 'smithi106', 'smithi112'] 2022-02-01T19:49:18.906 INFO:teuthology.kill:2022-02-01 19:49:18,906.906 INFO:teuthology.nuke:targets: 2022-02-01T19:49:18.907 INFO:teuthology.kill: smithi068.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSVJQL+vqFdA3dTw5inxOQs7HQkdE2E8tfqkGirOrsSEiek91p7IqokPGv/K5zLKsl0W4VXnVzS7kMSR89W+RxIzHWvjKCUrpN4XofRjGx8QhADHUzT34keDLPs5gMb5Ht6L5XHHbc0iaMkBjbOr6NnRX2bFl3KaV0NSjmtKGK1y6U0paLPUv4AIPvdG1a6oGjBXS8fJjBQVNIOjO3buWQYTeWfcBXfYTIcsxzZfTiYkUR1W5T0yhAQbMjOiPD+FC5uSpCZbjdfDRYUw9mMaYdYcKpZyiigpAobY5MTjs12lLbsmmNQZT/MOuzAaO+mak9+kkznh5Q5NFNeCA/dFAKkd8cOjse7y7theFY6U5K+XL2ZOJRA6iT//cEKYv5IovEcDALtJHnhgcBgIszSGbSq/Iw9RDR453xhjoWOe9cwsxEtLbsnDZ85DaLJ1S9e7N1h/I/8y5/slOxuRjr6citHSyD9wkbudEhhP8y2IglN1+pn7ZFwyI5uj7dgLgUcTM= 2022-02-01T19:49:18.908 INFO:teuthology.kill: smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC92hvuLfWJ7gBOyBCBNTXldGppXaPNrgLM3e2I1KjXezLRpNELL1ZlBziz2ueMwPROrRz4XLdNCKy7SAmCHQH+utdHmzMCyRwvXQW7t2fw1iQZ2TNao5RR5F9lLZkMh3Afx0O3nem/A2uFrBi/SUM9B5PuhRH+amcIqVUUem9VOHWWWcMan6uBaZEa9wUvPTBd27jT2kBf9nCpaeJBVxgxiZIcpWkUOMVTiBtszEonIURuSSWH7PQKFdRbK4bFbOJeDE4r7iCwlXcUlj+jx8IbBt2Ze6TPlcSi6Hia5TwaoolGL4yA7ZHCJUTlaLxSfTJV2Ode8NHPocrUqUCCdE1xH/QaCBoDiJwAF0y3tc9loeZiWrM0QX1CTL+25MLkJlv0XSYU4SIdES78kh+G7JZJlgtCqBPP6dzgPixOyo4u+7ZEsG1rMiGaeXN6IfJLWy5/PhxQkNM8P3U8yz3itB81d9PvOX4Dxdsur4jwiDvJ/y1ynuYQ6L799ADiVZUFWFk= 2022-02-01T19:49:18.908 INFO:teuthology.kill: smithi112.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOnYqZsSfaPy+q/hSrYl6eVwURW6imp0jrMZeATr9yQ/hmOVYA2bYhQUhaqcsrWwhu5MxPF+4y5nNtIwBXv46puWG8WZzJWwpG1CiO6HFhysK+tqVolnRJPOrqiXaX6FFsYwTP3XTAT1C6rxUklSKRw6RBJ6MxOSfJP+OpalF4mw7hx3OkG4oBDrRjy243vXcS8q9piCtjQYtuk0KgFUvE4aDcYjjJeIiJPTrMmEXbywMIRrz5AtQ2PZW3pdwq/WcoTJm40V+k1z8D1PP3AU29hovc31GXBc5/nAD+yJYoHqXC1NFxVtIiJPuNWhwmKaVIS740YwGSKrIxBtd2D1NiT1LPLpPl1bWcSM5P8C++tZ/qwA5D4hgL9R1+x0C4tSjl3+i0Ww9h5I/iw7Fty+Y0E7A1MRG6naRdWkYaNYSIP4xTIe06cEX3lgGdoSNQ8oqGMpsGEMbLwWxHPy84Kh+ah+n/THH5K+CO0z3J03jfodMx2rcFNMkA3fF/7KtngSM= 2022-02-01T19:49:18.937 INFO:teuthology.kill:2022-02-01 19:49:18,937.937 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-01T19:49:18.942 INFO:teuthology.kill:2022-02-01 19:49:18,942.942 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-01T19:49:18.945 INFO:teuthology.kill:2022-02-01 19:49:18,945.945 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-01T19:49:19.024 INFO:teuthology.kill:2022-02-01 19:49:19,023.023 INFO:teuthology.orchestra.console:Power off smithi106 2022-02-01T19:49:19.040 INFO:teuthology.kill:2022-02-01 19:49:19,039.039 INFO:teuthology.orchestra.console:Power off smithi068 2022-02-01T19:49:19.051 INFO:teuthology.kill:2022-02-01 19:49:19,050.050 INFO:teuthology.orchestra.console:Power off smithi112 2022-02-01T19:49:27.412 INFO:teuthology.kill:2022-02-01 19:49:27,412.412 INFO:teuthology.orchestra.console:Power off for smithi068 completed 2022-02-01T19:49:27.418 INFO:teuthology.kill:2022-02-01 19:49:27,418.418 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2022-02-01T19:49:27.557 INFO:teuthology.kill:2022-02-01 19:49:27,557.557 INFO:teuthology.lock.ops:unlocked: smithi068.front.sepia.ceph.com 2022-02-01T19:49:27.565 INFO:teuthology.kill:2022-02-01 19:49:27,565.565 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2022-02-01T19:49:31.516 INFO:teuthology.kill:2022-02-01 19:49:31,515.515 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2022-02-01T19:49:31.660 INFO:teuthology.kill:2022-02-01 19:49:31,660.660 INFO:teuthology.lock.ops:unlocked: smithi106.front.sepia.ceph.com 2022-02-01T19:51:31.873 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15