2023-02-11T01:43:53.724 INFO:root:teuthology version: 0.0.1.dev37+g00690ec 2023-02-11T01:43:53.861 INFO:teuthology.lock.ops:Start node 'smithi172.front.sepia.ceph.com' reimaging 2023-02-11T01:43:53.861 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: reset os type and version on server 2023-02-11T01:43:53.862 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-02-11T01:43:53.878 INFO:teuthology.lock.ops:Node 'smithi172.front.sepia.ceph.com' reimaging is complete 2023-02-11T01:43:53.878 INFO:teuthology.lock.ops:Start node 'smithi175.front.sepia.ceph.com' reimaging 2023-02-11T01:43:53.878 INFO:teuthology.lock.ops:Updating [smithi175.front.sepia.ceph.com]: reset os type and version on server 2023-02-11T01:43:53.878 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2023-02-11T01:43:53.895 INFO:teuthology.lock.ops:Node 'smithi175.front.sepia.ceph.com' reimaging is complete 2023-02-11T01:43:54.030 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of rhel 8.6 2023-02-11T01:43:54.049 INFO:teuthology.provision.fog.smithi175:Scheduling deploy of rhel 8.6 2023-02-11T01:43:54.272 INFO:teuthology.orchestra.console:Power off smithi175 2023-02-11T01:43:54.439 INFO:teuthology.orchestra.console:Power off smithi172 2023-02-11T01:44:03.393 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2023-02-11T01:44:03.413 INFO:teuthology.orchestra.console:Power off for smithi175 completed 2023-02-11T01:44:03.514 INFO:teuthology.orchestra.console:Power on smithi172 2023-02-11T01:44:03.524 INFO:teuthology.orchestra.console:Power on smithi175 2023-02-11T01:44:11.894 INFO:teuthology.orchestra.console:Power on for smithi175 completed 2023-02-11T01:44:11.972 INFO:teuthology.orchestra.console:Power on for smithi172 completed 2023-02-11T01:44:12.072 INFO:teuthology.provision.fog.smithi175:Waiting for deploy to finish 2023-02-11T01:44:12.115 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2023-02-11T01:46:46.053 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.175 2023-02-11T01:46:56.005 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.175 2023-02-11T01:47:01.134 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:47:05.097 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.175 2023-02-11T01:47:10.213 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:47:14.149 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.175 2023-02-11T01:47:19.269 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:47:23.241 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.175 2023-02-11T01:47:28.361 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:47:32.329 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.175 2023-02-11T01:47:37.445 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:47:46.565 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:47:55.653 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:48:04.745 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:48:29.317 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:48:38.387 WARNING:teuthology.provision.fog:timed out 2023-02-11T01:48:38.405 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-02-11T01:48:44.746 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-11T01:48:45.038 INFO:teuthology.provision.fog.smithi175:Node is ready 2023-02-11T01:48:45.055 INFO:teuthology.orchestra.run.smithi175.stdout:smithi175.front.sepia.ceph.com 2023-02-11T01:48:45.113 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175 smithi175.front.sepia.ceph.com smithi175 2023-02-11T01:48:45.460 INFO:teuthology.provision.fog.smithi175:Deploy complete! 2023-02-11T01:48:46.383 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-11T01:48:46.779 INFO:teuthology.provision.fog.smithi172:Node is ready 2023-02-11T01:48:46.795 INFO:teuthology.orchestra.run.smithi172.stdout:smithi022 2023-02-11T01:48:47.058 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2023-02-11T01:48:47.293 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2023-02-11T01:48:47.295 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-11T01:48:47.313 INFO:teuthology.lock.ops:Checking smithi175.front.sepia.ceph.com 2023-02-11T01:48:47.325 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-11T01:48:47.341 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2023-02-11T01:48:47.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-02-11T01:48:47.530 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2023-02-11T01:48:47.587 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Red Hat Enterprise Linux" 2023-02-11T01:48:47.588 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8.6 (Ootpa)" 2023-02-11T01:48:47.588 INFO:teuthology.orchestra.run.smithi172.stdout:ID="rhel" 2023-02-11T01:48:47.588 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="fedora" 2023-02-11T01:48:47.588 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8.6" 2023-02-11T01:48:47.588 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.redhat.com/" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-02-11T01:48:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-02-11T01:48:47.591 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-02-11T01:48:47.610 INFO:teuthology.lock.ops:Updating [smithi175.front.sepia.ceph.com]: set os type and version on server 2023-02-11T01:48:47.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-02-11T01:48:47.808 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2023-02-11T01:48:47.869 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Red Hat Enterprise Linux" 2023-02-11T01:48:47.907 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="8.6 (Ootpa)" 2023-02-11T01:48:47.907 INFO:teuthology.orchestra.run.smithi175.stdout:ID="rhel" 2023-02-11T01:48:47.907 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="fedora" 2023-02-11T01:48:47.907 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="8.6" 2023-02-11T01:48:47.908 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el8" 2023-02-11T01:48:47.908 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-02-11T01:48:47.908 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2023-02-11T01:48:47.908 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-02-11T01:48:47.908 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.redhat.com/" 2023-02-11T01:48:47.908 INFO:teuthology.orchestra.run.smithi175.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-02-11T01:48:47.909 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-11T01:48:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-02-11T01:48:47.909 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-11T01:48:47.909 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-02-11T01:48:47.909 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-02-11T01:48:47.909 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-02-11T01:48:47.910 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2023-02-11T01:48:48.013 INFO:teuthology.dispatcher.supervisor:Running job 7160000 2023-02-11T01:48:48.013 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_00690eca1aa0bd56527bb28e9f9e35d734d406bf/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160000 --name yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi --description rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{rhel_8} tasks/progress} -- /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160000/orig.config.yaml 2023-02-11T01:48:48.022 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160000 2023-02-11T01:48:48.022 INFO:teuthology.dispatcher.supervisor:Job PID: 11715 2023-02-11T01:48:48.022 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-02-11T13:49:53.295 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-02-11T13:50:00.183 INFO:teuthology.kill:Killing Pids: {11715} 2023-02-11T13:50:06.892 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'] 2023-02-11T13:50:06.893 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1'] 2023-02-11T13:50:06.899 INFO:teuthology.misc:Compressing logs... 2023-02-11T13:50:06.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-02-11T13:50:09.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-02-11T13:50:10.711 INFO:teuthology.kill:No teuthology processes running 2023-02-11T13:50:11.804 INFO:teuthology.kill:Nuking machines: ['smithi172', 'smithi175'] 2023-02-11T13:50:13.102 INFO:teuthology.kill:2023-02-11 13:50:13,102.102 INFO:teuthology.nuke:targets: 2023-02-11T13:50:13.103 INFO:teuthology.kill: smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= 2023-02-11T13:50:13.105 INFO:teuthology.kill: smithi175.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= 2023-02-11T13:50:13.105 INFO:teuthology.kill:2023-02-11 13:50:13,103.103 INFO:teuthology.nuke:Checking targets against current locks 2023-02-11T13:50:13.149 INFO:teuthology.kill:2023-02-11 13:50:13,149.149 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-11T13:50:13.170 INFO:teuthology.kill:2023-02-11 13:50:13,170.170 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-11T13:50:13.223 INFO:teuthology.kill:2023-02-11 13:50:13,222.222 INFO:teuthology.orchestra.console:Power off smithi172 2023-02-11T13:50:13.293 INFO:teuthology.kill:2023-02-11 13:50:13,292.292 INFO:teuthology.orchestra.console:Power off smithi175 2023-02-11T13:50:26.246 INFO:teuthology.kill:2023-02-11 13:50:26,245.245 INFO:teuthology.orchestra.console:Power off for smithi175 completed 2023-02-11T13:50:26.246 INFO:teuthology.kill:2023-02-11 13:50:26,246.246 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2023-02-11T13:50:26.418 INFO:teuthology.kill:2023-02-11 13:50:26,418.418 INFO:teuthology.lock.ops:unlocked: smithi175.front.sepia.ceph.com 2023-02-11T13:50:26.493 INFO:teuthology.kill:2023-02-11 13:50:26,493.493 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com 2023-02-11T13:52:27.070 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-02-11T13:52:27.392 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi172 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-02-08_19:43:15-rados-wip-yuri3-testing-2023-02-07-0852-quincy-distro-default-smithi/7160857