2023-02-03T21:06:28.775 INFO:root:teuthology version: 0.0.1.dev37+g00690ec 2023-02-03T21:06:28.893 INFO:teuthology.lock.ops:Start node 'smithi026.front.sepia.ceph.com' reimaging 2023-02-03T21:06:28.894 INFO:teuthology.lock.ops:Updating [smithi026.front.sepia.ceph.com]: reset os type and version on server 2023-02-03T21:06:28.894 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2023-02-03T21:06:28.911 INFO:teuthology.lock.ops:Node 'smithi026.front.sepia.ceph.com' reimaging is complete 2023-02-03T21:06:28.911 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging 2023-02-03T21:06:28.912 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server 2023-02-03T21:06:28.912 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-02-03T21:06:28.928 INFO:teuthology.lock.ops:Node 'smithi098.front.sepia.ceph.com' reimaging is complete 2023-02-03T21:06:29.063 INFO:teuthology.provision.fog.smithi026:Scheduling deploy of centos 8.stream 2023-02-03T21:06:29.086 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 8.stream 2023-02-03T21:06:29.368 INFO:teuthology.orchestra.console:Power off smithi098 2023-02-03T21:06:29.383 INFO:teuthology.orchestra.console:Power off smithi026 2023-02-03T21:06:29.512 INFO:teuthology.orchestra.console:Power off for smithi098 completed 2023-02-03T21:06:29.525 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2023-02-03T21:06:29.614 INFO:teuthology.orchestra.console:Power on smithi098 2023-02-03T21:06:29.627 INFO:teuthology.orchestra.console:Power on smithi026 2023-02-03T21:06:37.978 INFO:teuthology.orchestra.console:Power on for smithi098 completed 2023-02-03T21:06:38.006 INFO:teuthology.orchestra.console:Power on for smithi026 completed 2023-02-03T21:06:38.080 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish 2023-02-03T21:06:38.107 INFO:teuthology.provision.fog.smithi026:Waiting for deploy to finish 2023-02-03T21:09:27.568 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-02-03T21:09:36.628 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-02-03T21:09:42.672 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-02-03T21:09:45.712 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-02-03T21:09:51.732 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-02-03T21:10:00.820 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-02-03T21:10:09.904 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-02-03T21:10:18.961 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-02-03T21:10:35.156 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-02-03T21:10:44.244 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-02-03T21:10:51.718 WARNING:teuthology.provision.fog:timed out 2023-02-03T21:10:53.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-02-03T21:10:58.031 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-03T21:10:58.277 INFO:teuthology.provision.fog.smithi026:Node is ready 2023-02-03T21:10:58.337 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026.front.sepia.ceph.com 2023-02-03T21:10:58.397 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26 smithi026.front.sepia.ceph.com smithi026 2023-02-03T21:10:58.759 INFO:teuthology.provision.fog.smithi026:Deploy complete! 2023-02-03T21:10:59.669 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-03T21:11:19.968 INFO:teuthology.provision.fog.smithi098:Node is ready 2023-02-03T21:11:20.029 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com 2023-02-03T21:11:20.088 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098 2023-02-03T21:11:20.479 INFO:teuthology.provision.fog.smithi098:Deploy complete! 2023-02-03T21:11:20.667 INFO:teuthology.lock.ops:Checking smithi026.front.sepia.ceph.com 2023-02-03T21:11:20.668 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-03T21:11:20.684 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com 2023-02-03T21:11:20.685 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-03T21:11:20.699 INFO:teuthology.lock.ops:Updating [smithi026.front.sepia.ceph.com]: set os type and version on server 2023-02-03T21:11:20.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-02-03T21:11:20.966 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2023-02-03T21:11:21.025 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2023-02-03T21:11:21.025 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8" 2023-02-03T21:11:21.025 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2023-02-03T21:11:21.026 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2023-02-03T21:11:21.026 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8" 2023-02-03T21:11:21.026 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2023-02-03T21:11:21.026 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 8" 2023-02-03T21:11:21.026 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2023-02-03T21:11:21.027 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-02-03T21:11:21.027 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2023-02-03T21:11:21.027 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-03T21:11:21.027 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-03T21:11:21.027 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-02-03T21:11:21.028 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2023-02-03T21:11:21.049 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server 2023-02-03T21:11:21.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-02-03T21:11:21.244 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2023-02-03T21:11:21.302 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2023-02-03T21:11:21.302 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8" 2023-02-03T21:11:21.303 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2023-02-03T21:11:21.303 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2023-02-03T21:11:21.304 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8" 2023-02-03T21:11:21.304 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-02-03T21:11:21.304 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 8" 2023-02-03T21:11:21.305 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-02-03T21:11:21.305 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-02-03T21:11:21.305 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2023-02-03T21:11:21.306 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-03T21:11:21.306 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-03T21:11:21.306 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-02-03T21:11:21.307 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-02-03T21:11:21.392 INFO:teuthology.dispatcher.supervisor:Running job 7146518 2023-02-03T21:11:21.393 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_00690eca1aa0bd56527bb28e9f9e35d734d406bf/virtualenv/bin/teuthology -v --owner scheduled_pdhange@teuthology --archive /home/teuthworker/archive/pdhange-2023-02-03_06:44:07-rados-wip-pdhange-testing-distro-default-smithi/7146518 --name pdhange-2023-02-03_06:44:07-rados-wip-pdhange-testing-distro-default-smithi --description rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} -- /home/teuthworker/archive/pdhange-2023-02-03_06:44:07-rados-wip-pdhange-testing-distro-default-smithi/7146518/orig.config.yaml 2023-02-03T21:11:21.396 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdhange-2023-02-03_06:44:07-rados-wip-pdhange-testing-distro-default-smithi/7146518 2023-02-03T21:11:21.397 INFO:teuthology.dispatcher.supervisor:Job PID: 939 2023-02-03T21:11:21.397 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-02-04T09:12:14.016 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-02-04T09:12:14.164 INFO:teuthology.kill:Killing Pids: {939} 2023-02-04T09:12:15.026 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2023-02-04T09:12:15.027 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['osd.3', 'osd.4', 'osd.5', 'client.0'] 2023-02-04T09:12:15.027 INFO:teuthology.misc:Compressing logs... 2023-02-04T09:12:15.028 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-02-04T09:12:15.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-02-04T09:12:15.526 INFO:teuthology.misc:Compressing logs... 2023-02-04T09:12:16.649 INFO:teuthology.orchestra.run.smithi098.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2023-02-04T09:12:16.717 INFO:teuthology.orchestra.run.smithi026.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2023-02-04T09:12:17.394 INFO:teuthology.misc:Compressing logs... 2023-02-04T09:14:06.486 INFO:teuthology.kill:No teuthology processes running 2023-02-04T09:14:07.223 INFO:teuthology.kill:Nuking machines: ['smithi026', 'smithi098'] 2023-02-04T09:14:07.857 INFO:teuthology.kill:2023-02-04 09:14:07,857.857 INFO:teuthology.nuke:targets: 2023-02-04T09:14:07.858 INFO:teuthology.kill: smithi026.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzIyFM1G+fJ+ZrUPTtUhS4wf0CBXixMKql1vzQd4um9JMwndfjPKkXJS+pldMpIMoLGwgAm6mJRFJtgJ2hpTADe4kVYEMaT00lA1IwNPVPwURr4xG4jMBUNJnHO7fENiP2y2B2K978g6NE3q1vSq8wUGIEEKF9SBsYBGijPc6KdcbVln7UPb4137749NdwGkjkB16XsiWNJic0tVIczvzhFA2u5nah0sFENIy+bsb65lnKl/ccldn/n1U6diPupwcdfP1XMc05qv6lLYZBPlE8Acg8Of5Iiw4quJolpjxdpGOQ4FFnonEUJQmP7GMcFP3b9XOBryZtVOVhu2ErMP/36+z4TbNgChEGgXBKyxExpQWXQZkNL0MCIQWXnFKzoyB/A79VH4uhIobaC1gHwgKpRiwOyA9/joovatnSN3Al3gRwMTtocBtkR6AedH65fgh2yCfet4w7J59vapph+clYL1U22XW2e/qOKXj8B/GIRNRDF3AaZY9dV6yQFCK8bgM= 2023-02-04T09:14:07.858 INFO:teuthology.kill: smithi098.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDn9lE3KEMrYXwjzbOVbbjabldBzBm5IP+I9opotoBLvhgk6BpGtUJTj8bC/69tcdAXstsTPzlMRPbsQxdPQveGVdmOnXbP/iB3ie1/hZLFPuPJb/l3KQ4asxaKB6KW9CQEes/2Us1g5bYSahOUmKnqGVzrLV6CnQgHH0DAUstJ0KOQ8gpjC5cVuHb0NoKFmEss9aFDCqZeOVR3JpnRXaoK9o1Q7SeqVxBnT6Irv44WboSs6yrv+6bzsXx4zuOHdWp39I+v90gYAMFmcvpqZcxQi7II0dqxL4E5MJKsGfYXJxzoaezzRcqOGJoCxxp6vvh9Y1GY9yn9u8x7yTmcKf2ZFQIfoqbFCAhu89V/QtFkWWQ7RJ8Cmk8zAFvKsI8AH8+NOadHHicTQh9j4zTX5+uyYX5vpm5dR+0ZXr/Q/CIT7DGquTYQKBS3q67HPhvx0Q25L+Iu32Iiq45lcZ1WenRZhRmlMZU+OI8XJYo+LYDETOt7qkX3zpGv2ljlmFH1FqU= 2023-02-04T09:14:07.859 INFO:teuthology.kill:2023-02-04 09:14:07,858.858 INFO:teuthology.nuke:Checking targets against current locks 2023-02-04T09:14:07.933 INFO:teuthology.kill:2023-02-04 09:14:07,930.930 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-04T09:14:07.944 INFO:teuthology.kill:2023-02-04 09:14:07,943.943 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-04T09:14:08.014 INFO:teuthology.kill:2023-02-04 09:14:08,014.014 INFO:teuthology.orchestra.console:Power off smithi026 2023-02-04T09:14:08.030 INFO:teuthology.kill:2023-02-04 09:14:08,030.030 INFO:teuthology.orchestra.console:Power off smithi098 2023-02-04T09:14:16.388 INFO:teuthology.kill:2023-02-04 09:14:16,387.387 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2023-02-04T09:14:16.528 INFO:teuthology.kill:2023-02-04 09:14:16,527.527 INFO:teuthology.lock.ops:unlocked: smithi026.front.sepia.ceph.com 2023-02-04T09:14:20.510 INFO:teuthology.kill:2023-02-04 09:14:20,509.509 INFO:teuthology.orchestra.console:Power off for smithi098 completed 2023-02-04T09:14:20.664 INFO:teuthology.kill:2023-02-04 09:14:20,664.664 INFO:teuthology.lock.ops:unlocked: smithi098.front.sepia.ceph.com 2023-02-04T09:16:20.969 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-02-04T09:16:21.064 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi026 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-02-03_23:44:47-fs-wip-yuri8-testing-2023-01-30-1510-quincy-distro-default-smithi/7148595 2023-02-04T09:16:21.065 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi098 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-02-03_23:44:47-fs-wip-yuri8-testing-2023-01-30-1510-quincy-distro-default-smithi/7148595