2022-05-19T16:52:45.982 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-19T16:52:46.208 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging 2022-05-19T16:52:46.209 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server 2022-05-19T16:52:46.209 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2022-05-19T16:52:46.240 INFO:teuthology.lock.ops:Node 'smithi016.front.sepia.ceph.com' reimaging is complete 2022-05-19T16:52:46.240 INFO:teuthology.lock.ops:Start node 'smithi111.front.sepia.ceph.com' reimaging 2022-05-19T16:52:46.241 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: reset os type and version on server 2022-05-19T16:52:46.241 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2022-05-19T16:52:46.264 INFO:teuthology.lock.ops:Node 'smithi111.front.sepia.ceph.com' reimaging is complete 2022-05-19T16:52:46.465 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of ubuntu 18.04 2022-05-19T16:52:46.486 INFO:teuthology.provision.fog.smithi111:Scheduling deploy of ubuntu 18.04 2022-05-19T16:52:46.827 INFO:teuthology.orchestra.console:Power off smithi016 2022-05-19T16:52:46.892 INFO:teuthology.orchestra.console:Power off smithi111 2022-05-19T16:52:55.229 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2022-05-19T16:52:55.290 INFO:teuthology.orchestra.console:Power off for smithi111 completed 2022-05-19T16:52:55.333 INFO:teuthology.orchestra.console:Power on smithi016 2022-05-19T16:52:55.392 INFO:teuthology.orchestra.console:Power on smithi111 2022-05-19T16:53:03.773 INFO:teuthology.orchestra.console:Power on for smithi016 completed 2022-05-19T16:53:03.794 INFO:teuthology.orchestra.console:Power on for smithi111 completed 2022-05-19T16:53:03.876 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2022-05-19T16:53:03.895 INFO:teuthology.provision.fog.smithi111:Waiting for deploy to finish 2022-05-19T16:57:41.954 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-19T16:57:42.199 INFO:teuthology.provision.fog.smithi016:Node is ready 2022-05-19T16:57:42.246 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com 2022-05-19T16:57:42.301 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016 2022-05-19T16:57:42.645 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2022-05-19T16:57:55.675 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-05-19T16:58:01.691 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-05-19T16:58:07.708 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-05-19T16:58:13.722 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-05-19T16:58:19.737 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2022-05-19T16:58:25.843 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-19T16:58:26.106 INFO:teuthology.provision.fog.smithi111:Node is ready 2022-05-19T16:58:26.153 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111.front.sepia.ceph.com 2022-05-19T16:58:26.202 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111 smithi111.front.sepia.ceph.com smithi111 2022-05-19T16:58:26.528 INFO:teuthology.provision.fog.smithi111:Deploy complete! 2022-05-19T16:58:26.745 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2022-05-19T16:58:26.746 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-19T16:58:26.783 INFO:teuthology.lock.ops:Checking smithi111.front.sepia.ceph.com 2022-05-19T16:58:26.784 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-19T16:58:26.812 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2022-05-19T16:58:26.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2022-05-19T16:58:27.175 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2022-05-19T16:58:27.222 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Ubuntu" 2022-05-19T16:58:27.223 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2022-05-19T16:58:27.223 INFO:teuthology.orchestra.run.smithi016.stdout:ID=ubuntu 2022-05-19T16:58:27.224 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE=debian 2022-05-19T16:58:27.224 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2022-05-19T16:58:27.225 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="18.04" 2022-05-19T16:58:27.225 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.ubuntu.com/" 2022-05-19T16:58:27.226 INFO:teuthology.orchestra.run.smithi016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-05-19T16:58:27.227 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-05-19T16:58:27.228 INFO:teuthology.orchestra.run.smithi016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-05-19T16:58:27.228 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_CODENAME=bionic 2022-05-19T16:58:27.228 INFO:teuthology.orchestra.run.smithi016.stdout:UBUNTU_CODENAME=bionic 2022-05-19T16:58:27.230 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2022-05-19T16:58:27.251 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: set os type and version on server 2022-05-19T16:58:27.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2022-05-19T16:58:27.523 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2022-05-19T16:58:27.573 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="Ubuntu" 2022-05-19T16:58:27.574 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2022-05-19T16:58:27.574 INFO:teuthology.orchestra.run.smithi111.stdout:ID=ubuntu 2022-05-19T16:58:27.574 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE=debian 2022-05-19T16:58:27.574 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2022-05-19T16:58:27.575 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="18.04" 2022-05-19T16:58:27.575 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://www.ubuntu.com/" 2022-05-19T16:58:27.575 INFO:teuthology.orchestra.run.smithi111.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-05-19T16:58:27.575 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-05-19T16:58:27.575 INFO:teuthology.orchestra.run.smithi111.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-05-19T16:58:27.576 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_CODENAME=bionic 2022-05-19T16:58:27.576 INFO:teuthology.orchestra.run.smithi111.stdout:UBUNTU_CODENAME=bionic 2022-05-19T16:58:27.576 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2022-05-19T16:58:27.671 INFO:teuthology.dispatcher.supervisor:Running job 6841406 2022-05-19T16:58:27.672 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-05-19_16:47:41-rados:cephadm:osds-wip-lrc-fix-pacific-distro-default-smithi/6841406 --name lflores-2022-05-19_16:47:41-rados:cephadm:osds-wip-lrc-fix-pacific-distro-default-smithi --description rados:cephadm:osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} -- /home/teuthworker/archive/lflores-2022-05-19_16:47:41-rados:cephadm:osds-wip-lrc-fix-pacific-distro-default-smithi/6841406/orig.config.yaml 2022-05-19T16:58:27.677 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-05-19_16:47:41-rados:cephadm:osds-wip-lrc-fix-pacific-distro-default-smithi/6841406 2022-05-19T16:58:27.678 INFO:teuthology.dispatcher.supervisor:Job PID: 2175 2022-05-19T16:58:27.678 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-05-19T23:28:40.352 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-05-19T23:28:40.472 INFO:teuthology.kill:Killing Pids: {2175} 2022-05-19T23:28:41.986 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-05-19T23:28:41.987 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-05-19T23:28:41.988 INFO:teuthology.misc:Compressing logs... 2022-05-19T23:28:41.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2022-05-19T23:28:42.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2022-05-19T23:28:42.262 INFO:teuthology.orchestra.run.smithi016.stderr:find: '/var/lib/ceph/crash': No such file or directory 2022-05-19T23:28:42.596 INFO:teuthology.orchestra.run.smithi111.stderr:find: '/var/lib/ceph/crash': No such file or directory 2022-05-19T23:28:42.619 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-05-19T23:28:42.620 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2022-05-19T23:28:42.655 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-05-19T23:28:42.656 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2022-05-19T23:28:42.657 INFO:teuthology.misc:Compressing logs... 2022-05-19T23:28:42.896 INFO:teuthology.misc:Compressing logs... 2022-05-19T23:28:44.691 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-05-19T23:28:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-mgr.smithi016.fwnqeu.log: file size changed while zipping 2022-05-19T23:28:46.456 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-05-19T23:28:47.264 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-05-19T23:28:48.381 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-05-19T23:28:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-05-19T23:28:51.781 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-mon.smithi111.log: file size changed while zipping 2022-05-19T23:28:53.518 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-05-19T23:28:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-mon.smithi016.log: file size changed while zipping 2022-05-19T23:29:01.064 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/8d4ba8f0-d795-11ec-be40-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-05-19T23:29:07.195 INFO:teuthology.kill:No teuthology processes running 2022-05-19T23:29:08.617 INFO:teuthology.kill:Nuking machines: ['smithi016', 'smithi111'] 2022-05-19T23:29:09.880 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-05-19T23:29:09.881 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-05-19T23:29:09.881 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated 2022-05-19T23:29:09.882 INFO:teuthology.kill: "class": algorithms.Blowfish, 2022-05-19T23:29:09.882 INFO:teuthology.kill:2022-05-19 23:29:09,880.880 INFO:teuthology.nuke:targets: 2022-05-19T23:29:09.882 INFO:teuthology.kill: smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqOZMBlqOxpRMcYvfDHuM4uPpPhhVUMdB+LDsExGnuDKsMdVojVNubyrDghdNkh//GHxvQFD0/fcC7lvCRQXWdK+TZC6DrOXVdE1xMNadGfLvxS3/bjkRg9a+oJ1uFPxBxnFgLvBTvt3brGgFC/ldrQXpQk2qgN/C/ngD8SAt12w3DBsDoWoK8DPMhQf57DWiGsmM/N97gjR5aSpbdy/Tnv8Vp9+C+2psYbNBzF7kndACEL1ER3uSai58qjzvTCABBzKfsXZsvsT6T9I0Y8QWBE7yxvaLAoicgxgJNJ0ib4R2TCL4ggujw27JN0U0gzLPffNLGH9EPAnduVCaB6UHB 2022-05-19T23:29:09.883 INFO:teuthology.kill: smithi111.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDI+RCgq6WMxZntFT76xzTx7/15HNTgDLPB3FvlH0TEvAP0rtICjafwTtLV+xtz64QHpsbKIjDz+mTAjfidGQiFztyIjNt92yiF64szPoiiPmdkxrxKF5nmQBKNrmfOEO8ylhNOo2HJDsp95tgVLnjOzztleoDQdoCaaxXmZZl8iBIW7X39PxECXRSMzDJjwDjaK3HU4B3eBNzOnK1mW5MPBUST7PM3qM1tkQeCJW6q+MVmoBbFFo3EDqxpn52tSq0DLGTzandiMthbRJ04UcDMFnoKsJU9zbvyioRebg+u/OloSYSx3OCDEyBkJVg3jhItvjViXNf+lKTJsiEEKoeL 2022-05-19T23:29:09.913 INFO:teuthology.kill:2022-05-19 23:29:09,913.913 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-19T23:29:09.914 INFO:teuthology.kill:2022-05-19 23:29:09,914.914 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-19T23:29:10.003 INFO:teuthology.kill:2022-05-19 23:29:10,003.003 INFO:teuthology.orchestra.console:Power off smithi111 2022-05-19T23:29:10.027 INFO:teuthology.kill:2022-05-19 23:29:10,027.027 INFO:teuthology.orchestra.console:Power off smithi016 2022-05-19T23:29:18.384 INFO:teuthology.kill:2022-05-19 23:29:18,383.383 INFO:teuthology.orchestra.console:Power off for smithi111 completed 2022-05-19T23:29:18.536 INFO:teuthology.kill:2022-05-19 23:29:18,535.535 INFO:teuthology.lock.ops:unlocked: smithi111.front.sepia.ceph.com 2022-05-19T23:29:22.505 INFO:teuthology.kill:2022-05-19 23:29:22,503.503 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2022-05-19T23:29:22.660 INFO:teuthology.kill:2022-05-19 23:29:22,660.660 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com 2022-05-19T23:31:23.074 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-05-19T23:31:23.159 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi111 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-17_04:17:02-fs-pacific-distro-default-smithi/6837589