2022-10-01T04:41:49.495 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-01T04:41:49.654 INFO:teuthology.lock.ops:Start node 'smithi143.front.sepia.ceph.com' reimaging 2022-10-01T04:41:49.655 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: reset os type and version on server 2022-10-01T04:41:49.656 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2022-10-01T04:41:49.675 INFO:teuthology.lock.ops:Node 'smithi143.front.sepia.ceph.com' reimaging is complete 2022-10-01T04:41:49.676 INFO:teuthology.lock.ops:Start node 'smithi176.front.sepia.ceph.com' reimaging 2022-10-01T04:41:49.676 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: reset os type and version on server 2022-10-01T04:41:49.676 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2022-10-01T04:41:49.699 INFO:teuthology.lock.ops:Node 'smithi176.front.sepia.ceph.com' reimaging is complete 2022-10-01T04:41:49.896 INFO:teuthology.provision.fog.smithi143:Scheduling deploy of ubuntu 20.04 2022-10-01T04:41:49.914 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of ubuntu 20.04 2022-10-01T04:41:50.200 INFO:teuthology.orchestra.console:Power off smithi143 2022-10-01T04:41:50.220 INFO:teuthology.orchestra.console:Power off smithi176 2022-10-01T04:41:50.351 INFO:teuthology.orchestra.console:Power off for smithi143 completed 2022-10-01T04:41:50.361 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2022-10-01T04:41:50.452 INFO:teuthology.orchestra.console:Power on smithi143 2022-10-01T04:41:50.462 INFO:teuthology.orchestra.console:Power on smithi176 2022-10-01T04:41:58.825 INFO:teuthology.orchestra.console:Power on for smithi143 completed 2022-10-01T04:41:58.830 INFO:teuthology.orchestra.console:Power on for smithi176 completed 2022-10-01T04:41:58.927 INFO:teuthology.provision.fog.smithi143:Waiting for deploy to finish 2022-10-01T04:41:58.931 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish 2022-10-01T04:46:49.172 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-01T04:46:49.634 INFO:teuthology.provision.fog.smithi143:Node is ready 2022-10-01T04:46:49.641 INFO:teuthology.orchestra.run.smithi143.stdout:smithi143.front.sepia.ceph.com 2022-10-01T04:46:49.692 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143 smithi143.front.sepia.ceph.com smithi143 2022-10-01T04:46:49.842 INFO:teuthology.provision.fog.smithi143:Deploy complete! 2022-10-01T04:46:58.164 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-01T04:46:58.636 INFO:teuthology.provision.fog.smithi176:Node is ready 2022-10-01T04:46:58.643 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com 2022-10-01T04:46:58.651 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176 2022-10-01T04:46:58.811 INFO:teuthology.provision.fog.smithi176:Deploy complete! 2022-10-01T04:46:59.130 INFO:teuthology.lock.ops:Checking smithi143.front.sepia.ceph.com 2022-10-01T04:46:59.131 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-01T04:46:59.155 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com 2022-10-01T04:46:59.156 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-01T04:46:59.179 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: set os type and version on server 2022-10-01T04:46:59.180 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-10-01T04:46:59.410 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2022-10-01T04:46:59.418 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Ubuntu" 2022-10-01T04:46:59.418 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-10-01T04:46:59.419 INFO:teuthology.orchestra.run.smithi143.stdout:ID=ubuntu 2022-10-01T04:46:59.419 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE=debian 2022-10-01T04:46:59.419 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-10-01T04:46:59.420 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="20.04" 2022-10-01T04:46:59.420 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.ubuntu.com/" 2022-10-01T04:46:59.420 INFO:teuthology.orchestra.run.smithi143.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-10-01T04:46:59.420 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-10-01T04:46:59.420 INFO:teuthology.orchestra.run.smithi143.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-10-01T04:46:59.421 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_CODENAME=focal 2022-10-01T04:46:59.421 INFO:teuthology.orchestra.run.smithi143.stdout:UBUNTU_CODENAME=focal 2022-10-01T04:46:59.422 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2022-10-01T04:46:59.448 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server 2022-10-01T04:46:59.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2022-10-01T04:46:59.666 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2022-10-01T04:46:59.671 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Ubuntu" 2022-10-01T04:46:59.672 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-10-01T04:46:59.672 INFO:teuthology.orchestra.run.smithi176.stdout:ID=ubuntu 2022-10-01T04:46:59.672 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE=debian 2022-10-01T04:46:59.672 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-10-01T04:46:59.673 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="20.04" 2022-10-01T04:46:59.673 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.ubuntu.com/" 2022-10-01T04:46:59.673 INFO:teuthology.orchestra.run.smithi176.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-10-01T04:46:59.673 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-10-01T04:46:59.674 INFO:teuthology.orchestra.run.smithi176.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-10-01T04:46:59.674 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_CODENAME=focal 2022-10-01T04:46:59.674 INFO:teuthology.orchestra.run.smithi176.stdout:UBUNTU_CODENAME=focal 2022-10-01T04:46:59.675 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2022-10-01T04:46:59.798 INFO:teuthology.dispatcher.supervisor:Running job 7050792 2022-10-01T04:46:59.799 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-09-30_21:47:41-rados-wip-lflores-testing-distro-default-smithi/7050792 --name lflores-2022-09-30_21:47:41-rados-wip-lflores-testing-distro-default-smithi --description rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} -- /home/teuthworker/archive/lflores-2022-09-30_21:47:41-rados-wip-lflores-testing-distro-default-smithi/7050792/orig.config.yaml 2022-10-01T04:46:59.803 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-09-30_21:47:41-rados-wip-lflores-testing-distro-default-smithi/7050792 2022-10-01T04:46:59.804 INFO:teuthology.dispatcher.supervisor:Job PID: 3830 2022-10-01T04:46:59.805 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-10-01T16:47:57.203 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-10-01T16:47:57.380 INFO:teuthology.kill:Killing Pids: {3830} 2022-10-01T16:47:58.765 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2022-10-01T16:47:58.766 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.b', 'mds.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2022-10-01T16:47:58.767 INFO:teuthology.misc:Compressing logs... 2022-10-01T16:47:58.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-10-01T16:47:59.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2022-10-01T16:47:59.319 INFO:teuthology.misc:Compressing logs... 2022-10-01T16:47:59.490 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-10-01T16:47:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-10-01T16:47:59.745 INFO:teuthology.misc:Compressing logs... 2022-10-01T16:48:32.764 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-10-01T16:48:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-10-01T16:48:57.306 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-10-01T16:49:00.757 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-10-01T16:49:03.749 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-mgr.y.log: file size changed while zipping 2022-10-01T16:49:16.449 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-10-01T16:49:30.596 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-10-01T16:49:57.182 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-10-01T16:49:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-10-01T16:50:04.718 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-10-01T16:50:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-10-01T16:50:53.033 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/71f53192-4145-11ed-8432-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-10-01T16:52:01.324 INFO:teuthology.kill:No teuthology processes running 2022-10-01T16:52:02.768 INFO:teuthology.kill:Nuking machines: ['smithi143', 'smithi176'] 2022-10-01T16:52:03.958 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-10-01T16:52:03.959 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-10-01T16:52:03.959 INFO:teuthology.kill:2022-10-01 16:52:03,957.957 INFO:teuthology.nuke:targets: 2022-10-01T16:52:03.959 INFO:teuthology.kill: smithi143.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCus8Mzhr+lo6GD4JdXvYs1xidU8xxFMh6MKsGw/uLTIsVVDFGq+JlwK3FsGC0J7kTU5tRfYfOCzbY9aRyxeNyFtz9+WM7kR4ythRZ6SbkK7A2Tg/4y+3unzPPZaHXxF741p11Yoow9y5k+8MML5WZo6pnyLILCtM3+PTC4M6JYg9MuI+xVNLSIIQVV385bqbxG4pnEJ+N2K8ELXTdoS7gPmLltqfMolEPZRzSbsJ68iW1PMhqWGJKLBX+nP3/K3VFGX4NeNY66ZQOlBRaLrKAK+gBdE2gXbJtdeobeUdgKerWhuVt8s+BSehyBB37r5TZYZvcmcUGx4g+nAV4JmeZL2NgCerKGPQ/lqoiScDsQgXH328axnxXQY3IzZAtY6dWiMWUDlqpAfB48tek9OQ5W4z6I1NER++uhJqujo2/3oWI7MNzgspbfxvcEbYtzdAGzCC3bhLkkOYMAgFTYVYLG2zFUO3sBbADr313g1bIo7zMwb+gxWGuYunxZSkXcbQM= 2022-10-01T16:52:03.960 INFO:teuthology.kill: smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLEsQIqq8Xgbx2ntyWXPIa6pObRs22ofmY2CNgdo7d4c7QmAvIHQllkKMpBBMRNp2jDYHlqSTXwfC2xtlV2RBSgxIujeH8KIu2k6Trlies0au5/0royH1MADLcO6bYdB89R3WUN1nBHfxU49KoYbeILP/iW0tNVHohPRdSJ9QphttUbfPwKRTxPdYBqI2BWOvsORJnusRb3qQEts+JNAU39W/YTX70na4nUFCc15DtUWtQVd61yTPNOB5+FRZFPPXpthL5JZYnUUbTjXUgj2rkgT8l4Fn5Q8uTQH+/GUak1QVs88Z4n7KfgXUVAX9An8vG289ZBHxTkzlNsLV4bX6okW2Mm1MxBCFIlxH/0yxK5XKkSCkkzplnFyL663n/fa0b70RX5QwgiTUYfITdK4IZlJLlL0tV5XBZqBWwdGMJQrdXbppZ7he0u2sD81YLmu6URy3+SvwnMVVgsmIrqn67kcL0xHF+JJ7a1VXlnJV2FNgMueAbhNPgoyZvjohHcis= 2022-10-01T16:52:03.960 INFO:teuthology.kill:2022-10-01 16:52:03,959.959 INFO:teuthology.nuke:Checking targets against current locks 2022-10-01T16:52:04.012 INFO:teuthology.kill:2022-10-01 16:52:04,012.012 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-01T16:52:04.042 INFO:teuthology.kill:2022-10-01 16:52:04,042.042 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-01T16:52:04.098 INFO:teuthology.kill:2022-10-01 16:52:04,098.098 INFO:teuthology.orchestra.console:Power off smithi143 2022-10-01T16:52:04.139 INFO:teuthology.kill:2022-10-01 16:52:04,138.138 INFO:teuthology.orchestra.console:Power off smithi176 2022-10-01T16:52:12.486 INFO:teuthology.kill:2022-10-01 16:52:12,485.485 INFO:teuthology.orchestra.console:Power off for smithi143 completed 2022-10-01T16:52:12.513 INFO:teuthology.kill:2022-10-01 16:52:12,512.512 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2022-10-01T16:52:12.657 INFO:teuthology.kill:2022-10-01 16:52:12,656.656 INFO:teuthology.lock.ops:unlocked: smithi143.front.sepia.ceph.com 2022-10-01T16:52:12.682 INFO:teuthology.kill:2022-10-01 16:52:12,681.681 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com 2022-10-01T16:54:13.090 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-10-01T16:54:13.219 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi143 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-30_20:17:13-rados-wip-yuri2-testing-2022-09-30-1047-distro-default-smithi/7050482 2022-10-01T16:54:13.220 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi176 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-30_20:17:13-rados-wip-yuri2-testing-2022-09-30-1047-distro-default-smithi/7050481