2023-02-15T09:10:38.360 INFO:root:teuthology version: 0.0.1.dev41+gfbbadb5 2023-02-15T09:10:38.476 INFO:teuthology.lock.ops:Start node 'smithi120.front.sepia.ceph.com' reimaging 2023-02-15T09:10:38.477 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: reset os type and version on server 2023-02-15T09:10:38.478 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2023-02-15T09:10:38.494 INFO:teuthology.lock.ops:Node 'smithi120.front.sepia.ceph.com' reimaging is complete 2023-02-15T09:10:38.494 INFO:teuthology.lock.ops:Start node 'smithi129.front.sepia.ceph.com' reimaging 2023-02-15T09:10:38.495 INFO:teuthology.lock.ops:Updating [smithi129.front.sepia.ceph.com]: reset os type and version on server 2023-02-15T09:10:38.495 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2023-02-15T09:10:38.514 INFO:teuthology.lock.ops:Node 'smithi129.front.sepia.ceph.com' reimaging is complete 2023-02-15T09:10:38.635 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.6 2023-02-15T09:10:38.669 INFO:teuthology.provision.fog.smithi129:Scheduling deploy of rhel 8.6 2023-02-15T09:10:38.863 INFO:teuthology.orchestra.console:Power off smithi120 2023-02-15T09:10:38.877 INFO:teuthology.orchestra.console:Power off smithi129 2023-02-15T09:10:47.244 INFO:teuthology.orchestra.console:Power off for smithi129 completed 2023-02-15T09:10:47.345 INFO:teuthology.orchestra.console:Power on smithi129 2023-02-15T09:10:51.346 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2023-02-15T09:10:51.448 INFO:teuthology.orchestra.console:Power on smithi120 2023-02-15T09:10:55.703 INFO:teuthology.orchestra.console:Power on for smithi120 completed 2023-02-15T09:10:55.709 INFO:teuthology.orchestra.console:Power on for smithi129 completed 2023-02-15T09:10:55.805 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish 2023-02-15T09:10:55.811 INFO:teuthology.provision.fog.smithi129:Waiting for deploy to finish 2023-02-15T09:14:00.326 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-02-15T09:14:00.327 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-02-15T09:14:09.413 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-02-15T09:14:09.414 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-02-15T09:14:18.469 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-02-15T09:14:18.469 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-02-15T09:14:27.525 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-02-15T09:14:27.526 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-02-15T09:14:36.581 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-02-15T09:15:14.651 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-15T09:15:14.935 INFO:teuthology.provision.fog.smithi129:Node is ready 2023-02-15T09:15:14.951 INFO:teuthology.orchestra.run.smithi129.stdout:smithi129.front.sepia.ceph.com 2023-02-15T09:15:15.008 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129 smithi129.front.sepia.ceph.com smithi129 2023-02-15T09:15:15.360 INFO:teuthology.provision.fog.smithi129:Deploy complete! 2023-02-15T09:15:33.545 WARNING:teuthology.provision.fog:timed out 2023-02-15T09:15:39.875 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-15T09:15:40.136 INFO:teuthology.provision.fog.smithi120:Node is ready 2023-02-15T09:15:40.194 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com 2023-02-15T09:15:40.252 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120 2023-02-15T09:15:40.595 INFO:teuthology.provision.fog.smithi120:Deploy complete! 2023-02-15T09:15:40.763 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com 2023-02-15T09:15:40.764 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-15T09:15:40.781 INFO:teuthology.lock.ops:Checking smithi129.front.sepia.ceph.com 2023-02-15T09:15:40.781 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-15T09:15:40.798 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server 2023-02-15T09:15:40.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-02-15T09:15:40.986 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2023-02-15T09:15:41.042 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux" 2023-02-15T09:15:41.042 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.6 (Ootpa)" 2023-02-15T09:15:41.043 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel" 2023-02-15T09:15:41.043 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora" 2023-02-15T09:15:41.043 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.6" 2023-02-15T09:15:41.043 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8" 2023-02-15T09:15:41.043 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-02-15T09:15:41.043 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2023-02-15T09:15:41.043 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-02-15T09:15:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/" 2023-02-15T09:15:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-02-15T09:15:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-15T09:15:41.044 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-02-15T09:15:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-15T09:15:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-02-15T09:15:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-02-15T09:15:41.044 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-02-15T09:15:41.046 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2023-02-15T09:15:41.064 INFO:teuthology.lock.ops:Updating [smithi129.front.sepia.ceph.com]: set os type and version on server 2023-02-15T09:15:41.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2023-02-15T09:15:41.320 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2023-02-15T09:15:41.379 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="Red Hat Enterprise Linux" 2023-02-15T09:15:41.379 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="8.6 (Ootpa)" 2023-02-15T09:15:41.379 INFO:teuthology.orchestra.run.smithi129.stdout:ID="rhel" 2023-02-15T09:15:41.380 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="fedora" 2023-02-15T09:15:41.380 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="8.6" 2023-02-15T09:15:41.380 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el8" 2023-02-15T09:15:41.380 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-02-15T09:15:41.380 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2023-02-15T09:15:41.380 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-02-15T09:15:41.380 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://www.redhat.com/" 2023-02-15T09:15:41.380 INFO:teuthology.orchestra.run.smithi129.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-02-15T09:15:41.381 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-15T09:15:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: 2023-02-15T09:15:41.381 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-15T09:15:41.381 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-02-15T09:15:41.381 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-02-15T09:15:41.381 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-02-15T09:15:41.382 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2023-02-15T09:15:41.553 INFO:teuthology.dispatcher.supervisor:Running job 7174762 2023-02-15T09:15:41.554 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/virtualenv/bin/teuthology -v --owner scheduled_nmordech@teuthology --archive /home/teuthworker/archive/nmordech-2023-02-15_09:06:34-rados:singleton-main-distro-default-smithi/7174762 --name nmordech-2023-02-15_09:06:34-rados:singleton-main-distro-default-smithi --description rados:singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} -- /home/teuthworker/archive/nmordech-2023-02-15_09:06:34-rados:singleton-main-distro-default-smithi/7174762/orig.config.yaml 2023-02-15T09:15:41.558 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nmordech-2023-02-15_09:06:34-rados:singleton-main-distro-default-smithi/7174762 2023-02-15T09:15:41.559 INFO:teuthology.dispatcher.supervisor:Job PID: 18033 2023-02-15T09:15:41.559 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-02-15T21:16:40.777 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-02-15T21:16:40.922 INFO:teuthology.kill:Killing Pids: {18033} 2023-02-15T21:16:41.783 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2023-02-15T21:16:41.784 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['osd.3', 'osd.4', 'osd.5', 'client.0'] 2023-02-15T21:16:41.784 INFO:teuthology.misc:Compressing logs... 2023-02-15T21:16:41.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-02-15T21:16:41.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2023-02-15T21:16:42.347 INFO:teuthology.misc:Compressing logs... 2023-02-15T21:16:42.765 INFO:teuthology.orchestra.run.smithi129.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2023-02-15T21:16:43.031 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2023-02-15T21:16:43.381 INFO:teuthology.misc:Compressing logs... 2023-02-15T21:19:58.747 INFO:teuthology.kill:No teuthology processes running 2023-02-15T21:19:59.475 INFO:teuthology.kill:Nuking machines: ['smithi120', 'smithi129'] 2023-02-15T21:20:00.082 INFO:teuthology.kill:2023-02-15 21:20:00,082.082 INFO:teuthology.nuke:targets: 2023-02-15T21:20:00.083 INFO:teuthology.kill: smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= 2023-02-15T21:20:00.083 INFO:teuthology.kill: smithi129.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= 2023-02-15T21:20:00.084 INFO:teuthology.kill:2023-02-15 21:20:00,082.082 INFO:teuthology.nuke:Checking targets against current locks 2023-02-15T21:20:00.142 INFO:teuthology.kill:2023-02-15 21:20:00,142.142 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-15T21:20:00.166 INFO:teuthology.kill:2023-02-15 21:20:00,166.166 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-15T21:20:00.223 INFO:teuthology.kill:2023-02-15 21:20:00,222.222 INFO:teuthology.orchestra.console:Power off smithi120 2023-02-15T21:20:00.260 INFO:teuthology.kill:2023-02-15 21:20:00,259.259 INFO:teuthology.orchestra.console:Power off smithi129 2023-02-15T21:20:12.704 INFO:teuthology.kill:2023-02-15 21:20:12,704.704 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2023-02-15T21:20:12.737 INFO:teuthology.kill:2023-02-15 21:20:12,737.737 INFO:teuthology.orchestra.console:Power off for smithi129 completed 2023-02-15T21:20:12.852 INFO:teuthology.kill:2023-02-15 21:20:12,852.852 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com 2023-02-15T21:20:12.881 INFO:teuthology.kill:2023-02-15 21:20:12,880.880 INFO:teuthology.lock.ops:unlocked: smithi129.front.sepia.ceph.com 2023-02-15T21:22:13.183 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-02-15T21:22:13.266 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi120 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-02-15_20:55:38-fs-wip-yuri2-testing-2023-02-09-0842-quincy-distro-default-smithi/7175226 2023-02-15T21:22:13.266 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi129 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-02-15_20:55:38-fs-wip-yuri2-testing-2023-02-09-0842-quincy-distro-default-smithi/7175226