2023-03-03T08:23:59.815 INFO:root:teuthology version: 0.0.1.dev49+gc7c3cc8
2023-03-03T08:23:59.960 INFO:teuthology.lock.ops:Start node 'smithi084.front.sepia.ceph.com' reimaging
2023-03-03T08:23:59.961 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: reset os type and version on server
2023-03-03T08:23:59.961 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2023-03-03T08:23:59.983 INFO:teuthology.lock.ops:Node 'smithi084.front.sepia.ceph.com' reimaging is complete
2023-03-03T08:23:59.983 INFO:teuthology.lock.ops:Start node 'smithi145.front.sepia.ceph.com' reimaging
2023-03-03T08:23:59.984 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: reset os type and version on server
2023-03-03T08:23:59.984 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server
2023-03-03T08:24:00.008 INFO:teuthology.lock.ops:Node 'smithi145.front.sepia.ceph.com' reimaging is complete
2023-03-03T08:24:00.156 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of rhel 8.6
2023-03-03T08:24:00.160 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of rhel 8.6
2023-03-03T08:24:00.364 INFO:teuthology.orchestra.console:Power off smithi145
2023-03-03T08:24:00.381 INFO:teuthology.orchestra.console:Power off smithi084
2023-03-03T08:24:08.762 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2023-03-03T08:24:08.863 INFO:teuthology.orchestra.console:Power on smithi084
2023-03-03T08:24:12.866 INFO:teuthology.orchestra.console:Power off for smithi145 completed
2023-03-03T08:24:12.968 INFO:teuthology.orchestra.console:Power on smithi145
2023-03-03T08:24:17.339 INFO:teuthology.orchestra.console:Power on for smithi084 completed
2023-03-03T08:24:17.440 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish
2023-03-03T08:24:21.339 INFO:teuthology.orchestra.console:Power on for smithi145 completed
2023-03-03T08:24:21.440 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish
2023-03-03T08:26:55.141 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-03T08:27:04.197 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-03T08:27:06.277 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-03-03T08:27:13.253 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-03T08:27:15.333 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-03-03T08:27:22.309 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-03T08:27:24.389 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-03-03T08:27:31.365 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-03T08:27:33.449 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-03-03T08:27:40.425 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-03T08:27:42.533 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-03-03T08:27:49.509 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.145
2023-03-03T08:27:51.589 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-03-03T08:28:55.569 WARNING:teuthology.provision.fog:timed out
2023-03-03T08:28:57.596 WARNING:teuthology.provision.fog:timed out
2023-03-03T08:29:01.903 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-03T08:29:02.194 INFO:teuthology.provision.fog.smithi145:Node is ready
2023-03-03T08:29:02.212 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145
2023-03-03T08:29:02.268 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145
2023-03-03T08:29:02.617 INFO:teuthology.provision.fog.smithi145:Deploy complete!
2023-03-03T08:29:03.944 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-03T08:29:04.241 INFO:teuthology.provision.fog.smithi084:Node is ready
2023-03-03T08:29:04.259 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084
2023-03-03T08:29:04.316 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084
2023-03-03T08:29:04.665 INFO:teuthology.provision.fog.smithi084:Deploy complete!
2023-03-03T08:29:04.836 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com
2023-03-03T08:29:04.837 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-03T08:29:04.862 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com
2023-03-03T08:29:04.863 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-03T08:29:04.887 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server
2023-03-03T08:29:04.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com'
2023-03-03T08:29:05.034 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64
2023-03-03T08:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Red Hat Enterprise Linux"
2023-03-03T08:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8.6 (Ootpa)"
2023-03-03T08:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout:ID="rhel"
2023-03-03T08:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="fedora"
2023-03-03T08:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8.6"
2023-03-03T08:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8"
2023-03-03T08:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-03T08:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31"
2023-03-03T08:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-03T08:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.redhat.com/"
2023-03-03T08:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-03T08:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-03T08:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout:
2023-03-03T08:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-03T08:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-03T08:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-03T08:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-03T08:29:05.093 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2023-03-03T08:29:05.111 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server
2023-03-03T08:29:05.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-03T08:29:05.263 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64
2023-03-03T08:29:05.322 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Red Hat Enterprise Linux"
2023-03-03T08:29:05.322 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8.6 (Ootpa)"
2023-03-03T08:29:05.322 INFO:teuthology.orchestra.run.smithi145.stdout:ID="rhel"
2023-03-03T08:29:05.323 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="fedora"
2023-03-03T08:29:05.323 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8.6"
2023-03-03T08:29:05.323 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8"
2023-03-03T08:29:05.323 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-03T08:29:05.323 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31"
2023-03-03T08:29:05.323 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-03T08:29:05.323 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.redhat.com/"
2023-03-03T08:29:05.324 INFO:teuthology.orchestra.run.smithi145.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-03T08:29:05.324 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-03T08:29:05.324 INFO:teuthology.orchestra.run.smithi145.stdout:
2023-03-03T08:29:05.324 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-03T08:29:05.324 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-03T08:29:05.324 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-03T08:29:05.324 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-03T08:29:05.325 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server
2023-03-03T08:29:05.445 INFO:teuthology.dispatcher.supervisor:Running job 7192250
2023-03-03T08:29:05.446 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192250 --name vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi --description fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} -- /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192250/orig.config.yaml
2023-03-03T08:29:05.450 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192250
2023-03-03T08:29:05.450 INFO:teuthology.dispatcher.supervisor:Job PID: 7567
2023-03-03T08:29:05.451 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-03T20:30:00.893 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-03T20:30:01.049 INFO:teuthology.kill:Killing Pids: {7567}
2023-03-03T20:30:01.851 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2023-03-03T20:30:01.852 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2023-03-03T20:30:01.852 INFO:teuthology.misc:Compressing logs...
2023-03-03T20:30:01.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com'
2023-03-03T20:30:02.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com'
2023-03-03T20:30:02.317 INFO:teuthology.misc:Compressing logs...
2023-03-03T20:30:02.490 INFO:teuthology.misc:Compressing logs...
2023-03-03T20:32:33.340 INFO:teuthology.kill:No teuthology processes running
2023-03-03T20:32:34.124 INFO:teuthology.kill:Nuking machines: ['smithi084', 'smithi145']
2023-03-03T20:32:34.718 INFO:teuthology.kill:2023-03-03 20:32:34,717.717 INFO:teuthology.nuke:targets:
2023-03-03T20:32:34.719 INFO:teuthology.kill:  smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-03-03T20:32:34.720 INFO:teuthology.kill:  smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=
2023-03-03T20:32:34.720 INFO:teuthology.kill:2023-03-03 20:32:34,718.718 INFO:teuthology.nuke:Checking targets against current locks
2023-03-03T20:32:34.778 INFO:teuthology.kill:2023-03-03 20:32:34,777.777 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-03T20:32:34.807 INFO:teuthology.kill:2023-03-03 20:32:34,806.806 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-03T20:32:34.868 INFO:teuthology.kill:2023-03-03 20:32:34,868.868 INFO:teuthology.orchestra.console:Power off smithi084
2023-03-03T20:32:34.913 INFO:teuthology.kill:2023-03-03 20:32:34,913.913 INFO:teuthology.orchestra.console:Power off smithi145
2023-03-03T20:32:47.366 INFO:teuthology.kill:2023-03-03 20:32:47,366.366 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2023-03-03T20:32:47.399 INFO:teuthology.kill:2023-03-03 20:32:47,398.398 INFO:teuthology.orchestra.console:Power off for smithi145 completed
2023-03-03T20:32:47.530 INFO:teuthology.kill:2023-03-03 20:32:47,530.530 INFO:teuthology.lock.ops:unlocked: smithi084.front.sepia.ceph.com
2023-03-03T20:32:47.548 INFO:teuthology.kill:2023-03-03 20:32:47,548.548 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com
2023-03-03T20:34:47.899 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-03T20:34:48.027 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi084 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-03-03_17:39:09-rados-reef-distro-default-smithi/7192694
2023-03-03T20:34:48.028 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi145 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-03-03_17:40:38-orch-reef-distro-default-smithi/7192695