2022-10-05T17:10:13.365 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-05T17:10:13.495 INFO:teuthology.lock.ops:Start node 'smithi040.front.sepia.ceph.com' reimaging 2022-10-05T17:10:13.496 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: reset os type and version on server 2022-10-05T17:10:13.497 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2022-10-05T17:10:13.515 INFO:teuthology.lock.ops:Node 'smithi040.front.sepia.ceph.com' reimaging is complete 2022-10-05T17:10:13.516 INFO:teuthology.lock.ops:Start node 'smithi059.front.sepia.ceph.com' reimaging 2022-10-05T17:10:13.516 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: reset os type and version on server 2022-10-05T17:10:13.517 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-10-05T17:10:13.537 INFO:teuthology.lock.ops:Node 'smithi059.front.sepia.ceph.com' reimaging is complete 2022-10-05T17:10:13.721 INFO:teuthology.provision.fog.smithi040:Scheduling deploy of centos 8.stream 2022-10-05T17:10:13.755 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of centos 8.stream 2022-10-05T17:10:14.080 INFO:teuthology.orchestra.console:Power off smithi040 2022-10-05T17:10:14.116 INFO:teuthology.orchestra.console:Power off smithi059 2022-10-05T17:10:22.449 INFO:teuthology.orchestra.console:Power off for smithi040 completed 2022-10-05T17:10:22.550 INFO:teuthology.orchestra.console:Power on smithi040 2022-10-05T17:10:26.586 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-10-05T17:10:26.688 INFO:teuthology.orchestra.console:Power on smithi059 2022-10-05T17:10:30.910 INFO:teuthology.orchestra.console:Power on for smithi040 completed 2022-10-05T17:10:30.937 INFO:teuthology.orchestra.console:Power on for smithi059 completed 2022-10-05T17:10:31.012 INFO:teuthology.provision.fog.smithi040:Waiting for deploy to finish 2022-10-05T17:10:31.038 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2022-10-05T17:14:51.172 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-05T17:14:51.281 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-05T17:14:51.441 INFO:teuthology.provision.fog.smithi059:Node is ready 2022-10-05T17:14:51.460 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2022-10-05T17:14:51.520 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2022-10-05T17:14:51.542 INFO:teuthology.provision.fog.smithi040:Node is ready 2022-10-05T17:14:51.561 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040.front.sepia.ceph.com 2022-10-05T17:14:51.621 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40 smithi040.front.sepia.ceph.com smithi040 2022-10-05T17:14:51.885 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2022-10-05T17:14:51.970 INFO:teuthology.provision.fog.smithi040:Deploy complete! 2022-10-05T17:14:52.219 INFO:teuthology.lock.ops:Checking smithi040.front.sepia.ceph.com 2022-10-05T17:14:52.221 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-05T17:14:52.241 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2022-10-05T17:14:52.241 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-05T17:14:52.263 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: set os type and version on server 2022-10-05T17:14:52.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2022-10-05T17:14:52.424 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2022-10-05T17:14:52.482 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2022-10-05T17:14:52.482 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="8" 2022-10-05T17:14:52.482 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2022-10-05T17:14:52.483 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2022-10-05T17:14:52.483 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="8" 2022-10-05T17:14:52.483 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el8" 2022-10-05T17:14:52.483 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-05T17:14:52.484 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2022-10-05T17:14:52.484 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-05T17:14:52.484 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2022-10-05T17:14:52.484 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-05T17:14:52.484 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-05T17:14:52.485 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-05T17:14:52.486 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2022-10-05T17:14:52.505 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2022-10-05T17:14:52.506 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-10-05T17:14:52.715 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2022-10-05T17:14:52.774 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2022-10-05T17:14:52.774 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8" 2022-10-05T17:14:52.775 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2022-10-05T17:14:52.775 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2022-10-05T17:14:52.775 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8" 2022-10-05T17:14:52.775 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2022-10-05T17:14:52.776 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-05T17:14:52.776 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2022-10-05T17:14:52.776 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-05T17:14:52.776 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2022-10-05T17:14:52.777 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-05T17:14:52.777 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-05T17:14:52.778 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-05T17:14:52.778 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-10-05T17:14:52.886 INFO:teuthology.dispatcher.supervisor:Running job 7054960 2022-10-05T17:14:52.886 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-10-05_14:50:39-upgrade:quincy-x-main-distro-default-smithi/7054960 --name teuthology-2022-10-05_14:50:39-upgrade:quincy-x-main-distro-default-smithi --description upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} -- /home/teuthworker/archive/teuthology-2022-10-05_14:50:39-upgrade:quincy-x-main-distro-default-smithi/7054960/orig.config.yaml 2022-10-05T17:14:52.891 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-10-05_14:50:39-upgrade:quincy-x-main-distro-default-smithi/7054960 2022-10-05T17:14:52.892 INFO:teuthology.dispatcher.supervisor:Job PID: 2401 2022-10-05T17:14:52.892 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-10-06T05:15:48.771 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-10-06T05:15:48.964 INFO:teuthology.kill:Killing Pids: {2401} 2022-10-06T05:15:50.569 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2022-10-06T05:15:50.570 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2022-10-06T05:15:50.570 INFO:teuthology.misc:Compressing logs... 2022-10-06T05:15:50.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2022-10-06T05:15:50.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-10-06T05:15:51.330 INFO:teuthology.misc:Compressing logs... 2022-10-06T05:15:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-10-06T05:15:51.699 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-10-06T05:15:51.905 INFO:teuthology.misc:Compressing logs... 2022-10-06T05:15:53.769 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-10-06T05:15:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-10-06T05:15:55.691 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-10-06T05:15:57.506 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-10-06T05:15:59.582 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-10-06T05:16:00.395 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-10-06T05:16:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-10-06T05:16:03.539 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-10-06T05:16:04.871 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-10-06T05:16:05.112 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-10-06T05:16:06.100 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-mgr.y.log: file size changed while zipping 2022-10-06T05:16:08.984 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/2e4c5a8e-44d2-11ed-8433-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-10-06T05:19:11.605 INFO:teuthology.orchestra.run.smithi040.stderr:tar: .: file changed as we read it 2022-10-06T05:21:29.475 INFO:teuthology.kill:No teuthology processes running 2022-10-06T05:21:31.074 INFO:teuthology.kill:Nuking machines: ['smithi040', 'smithi059'] 2022-10-06T05:21:32.565 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-10-06T05:21:32.566 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-10-06T05:21:32.567 INFO:teuthology.kill:2022-10-06 05:21:32,565.565 INFO:teuthology.nuke:targets: 2022-10-06T05:21:32.567 INFO:teuthology.kill: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGvIqX3i3RmP4B2fldB2YK3m8FOBadLxagmX33mJd33bR0f87po92Nqo6Ur55Z02ffeWdCeLs82GwxpZrHna/48= 2022-10-06T05:21:32.567 INFO:teuthology.kill: smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxrTPja6BF2Sy3D7N41JDrxlZWPPYHP9HVtBV3XI3KMNB5tyQ+dDNYCgpgFOh7wYueSg1QD+K686RHmJlvYNgM6aJFMc9Cv1GetYLq71KI1OhPEp6LvjG96cHd5/PUP484P0AGnSMIztCSHmDuvVH4E5GFEz0POL1JDBrCyxHr/l2KdfRRQxZzfqkIyk366V36qU4YpVhSkRljlZku9auM/Qg2cr66P2KKDFO3uUq6aVHKUR20T8qOGg4fS6/bTzxuBGxRMBAuELJCVyCd0z1bReHbOIr0d/v8DX0X+UQPIF6YeFr4Ffm62mOP5N1RErU2wTWOsDfJi0DPi4vJrPzBDXyqHpjqnO3krTxXpGFHCklVzT4OrETT3dPYNxfOGWBPGfR/xuDgAqQ7TDwB500sURNcp3e/YIfi/f68hM1GwOGHNojc9LO0aoEDlDQoGVpxxjv00t5hXpeil+cqPB/4SGyuQJoipNaSGB0RTUQQ3fVsVQYuXokwpxheTgPIcz0= 2022-10-06T05:21:32.567 INFO:teuthology.kill:2022-10-06 05:21:32,566.566 INFO:teuthology.nuke:Checking targets against current locks 2022-10-06T05:21:32.623 INFO:teuthology.kill:2022-10-06 05:21:32,623.623 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-06T05:21:32.646 INFO:teuthology.kill:2022-10-06 05:21:32,646.646 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-06T05:21:32.705 INFO:teuthology.kill:2022-10-06 05:21:32,705.705 INFO:teuthology.orchestra.console:Power off smithi040 2022-10-06T05:21:32.750 INFO:teuthology.kill:2022-10-06 05:21:32,750.750 INFO:teuthology.orchestra.console:Power off smithi059 2022-10-06T05:21:41.098 INFO:teuthology.kill:2022-10-06 05:21:41,097.097 INFO:teuthology.orchestra.console:Power off for smithi040 completed 2022-10-06T05:21:41.126 INFO:teuthology.kill:2022-10-06 05:21:41,124.124 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-10-06T05:21:41.257 INFO:teuthology.kill:2022-10-06 05:21:41,255.255 INFO:teuthology.lock.ops:unlocked: smithi040.front.sepia.ceph.com 2022-10-06T05:21:41.284 INFO:teuthology.kill:2022-10-06 05:21:41,284.284 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2022-10-06T05:23:41.677 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-10-06T05:23:41.875 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi040 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-10-06_05:00:03-smoke-main-distro-default-smithi/7057026 2022-10-06T05:23:41.875 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi059 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-10-06_05:00:03-smoke-main-distro-default-smithi/7057026