2023-03-03T06:07:56.066 INFO:root:teuthology version: 0.0.1.dev49+gc7c3cc8
2023-03-03T06:07:56.210 INFO:teuthology.lock.ops:Start node 'smithi005.front.sepia.ceph.com' reimaging
2023-03-03T06:07:56.211 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: reset os type and version on server
2023-03-03T06:07:56.211 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2023-03-03T06:07:56.239 INFO:teuthology.lock.ops:Node 'smithi005.front.sepia.ceph.com' reimaging is complete
2023-03-03T06:07:56.239 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging
2023-03-03T06:07:56.240 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server
2023-03-03T06:07:56.240 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2023-03-03T06:07:56.264 INFO:teuthology.lock.ops:Node 'smithi191.front.sepia.ceph.com' reimaging is complete
2023-03-03T06:07:56.432 INFO:teuthology.provision.fog.smithi005:Scheduling deploy of rhel 8.6
2023-03-03T06:07:56.440 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of rhel 8.6
2023-03-03T06:07:56.686 INFO:teuthology.orchestra.console:Power off smithi191
2023-03-03T06:07:56.703 INFO:teuthology.orchestra.console:Power off smithi005
2023-03-03T06:08:05.072 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2023-03-03T06:08:05.174 INFO:teuthology.orchestra.console:Power on smithi191
2023-03-03T06:08:09.193 INFO:teuthology.orchestra.console:Power off for smithi005 completed
2023-03-03T06:08:09.294 INFO:teuthology.orchestra.console:Power on smithi005
2023-03-03T06:08:13.550 INFO:teuthology.orchestra.console:Power on for smithi005 completed
2023-03-03T06:08:13.608 INFO:teuthology.orchestra.console:Power on for smithi191 completed
2023-03-03T06:08:13.652 INFO:teuthology.provision.fog.smithi005:Waiting for deploy to finish
2023-03-03T06:08:13.709 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish
2023-03-03T06:10:44.701 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2023-03-03T06:11:02.533 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-03-03T06:11:11.589 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-03-03T06:11:20.645 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-03-03T06:11:29.701 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-03-03T06:11:38.757 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-03-03T06:11:45.706 WARNING:teuthology.provision.fog:timed out
2023-03-03T06:11:54.793 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2023-03-03T06:12:03.845 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2023-03-03T06:12:12.901 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2023-03-03T06:12:22.105 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-03T06:12:22.428 INFO:teuthology.provision.fog.smithi191:Node is ready
2023-03-03T06:12:22.444 INFO:teuthology.orchestra.run.smithi191.stdout:smithi022
2023-03-03T06:12:22.500 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-03-03T06:12:22.868 INFO:teuthology.provision.fog.smithi191:Deploy complete!
2023-03-03T06:12:44.761 WARNING:teuthology.provision.fog:timed out
2023-03-03T06:12:51.146 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-03T06:12:51.433 INFO:teuthology.provision.fog.smithi005:Node is ready
2023-03-03T06:12:51.453 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005.front.sepia.ceph.com
2023-03-03T06:12:51.513 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5 smithi005.front.sepia.ceph.com smithi005
2023-03-03T06:12:51.867 INFO:teuthology.provision.fog.smithi005:Deploy complete!
2023-03-03T06:12:52.097 INFO:teuthology.lock.ops:Checking smithi005.front.sepia.ceph.com
2023-03-03T06:12:52.098 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-03T06:12:52.126 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com
2023-03-03T06:12:52.126 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-03T06:12:52.144 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: set os type and version on server
2023-03-03T06:12:52.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com'
2023-03-03T06:12:52.381 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64
2023-03-03T06:12:52.439 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux"
2023-03-03T06:12:52.440 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.6 (Ootpa)"
2023-03-03T06:12:52.440 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel"
2023-03-03T06:12:52.440 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora"
2023-03-03T06:12:52.440 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.6"
2023-03-03T06:12:52.440 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8"
2023-03-03T06:12:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-03T06:12:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31"
2023-03-03T06:12:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-03T06:12:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/"
2023-03-03T06:12:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-03T06:12:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-03T06:12:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:
2023-03-03T06:12:52.441 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-03T06:12:52.442 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-03T06:12:52.442 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-03T06:12:52.442 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-03T06:12:52.443 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2023-03-03T06:12:52.476 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server
2023-03-03T06:12:52.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com'
2023-03-03T06:12:52.741 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64
2023-03-03T06:12:52.799 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux"
2023-03-03T06:12:52.799 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.6 (Ootpa)"
2023-03-03T06:12:52.799 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel"
2023-03-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora"
2023-03-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.6"
2023-03-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8"
2023-03-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31"
2023-03-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/"
2023-03-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi191.stdout:
2023-03-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-03T06:12:52.802 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-03T06:12:52.802 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-03T06:12:52.803 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2023-03-03T06:12:52.931 INFO:teuthology.dispatcher.supervisor:Running job 7192149
2023-03-03T06:12:52.932 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192149 --name vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} -- /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192149/orig.config.yaml
2023-03-03T06:12:52.937 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192149
2023-03-03T06:12:52.938 INFO:teuthology.dispatcher.supervisor:Job PID: 1542
2023-03-03T06:12:52.938 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-03T18:13:17.005 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-03T18:13:17.146 INFO:teuthology.kill:Killing Pids: {1542}
2023-03-03T18:13:17.953 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2']
2023-03-03T18:13:17.954 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']
2023-03-03T18:13:17.954 INFO:teuthology.misc:Compressing logs...
2023-03-03T18:13:17.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com'
2023-03-03T18:13:18.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com'
2023-03-03T18:13:18.457 INFO:teuthology.misc:Compressing logs...
2023-03-03T18:13:18.635 INFO:teuthology.misc:Compressing logs...
2023-03-03T18:16:09.351 INFO:teuthology.kill:No teuthology processes running
2023-03-03T18:16:10.096 INFO:teuthology.kill:Nuking machines: ['smithi005', 'smithi191']
2023-03-03T18:16:10.681 INFO:teuthology.kill:2023-03-03 18:16:10,680.680 INFO:teuthology.nuke:targets:
2023-03-03T18:16:10.682 INFO:teuthology.kill:  smithi005.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-03-03T18:16:10.682 INFO:teuthology.kill:  smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=
2023-03-03T18:16:10.682 INFO:teuthology.kill:2023-03-03 18:16:10,680.680 INFO:teuthology.nuke:Checking targets against current locks
2023-03-03T18:16:10.738 INFO:teuthology.kill:2023-03-03 18:16:10,738.738 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-03T18:16:10.766 INFO:teuthology.kill:2023-03-03 18:16:10,766.766 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-03T18:16:10.809 INFO:teuthology.kill:2023-03-03 18:16:10,808.808 INFO:teuthology.orchestra.console:Power off smithi005
2023-03-03T18:16:10.848 INFO:teuthology.kill:2023-03-03 18:16:10,848.848 INFO:teuthology.orchestra.console:Power off smithi191
2023-03-03T18:16:19.189 INFO:teuthology.kill:2023-03-03 18:16:19,189.189 INFO:teuthology.orchestra.console:Power off for smithi005 completed
2023-03-03T18:16:19.216 INFO:teuthology.kill:2023-03-03 18:16:19,216.216 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2023-03-03T18:16:19.354 INFO:teuthology.kill:2023-03-03 18:16:19,354.354 INFO:teuthology.lock.ops:unlocked: smithi005.front.sepia.ceph.com
2023-03-03T18:16:19.377 INFO:teuthology.kill:2023-03-03 18:16:19,376.376 INFO:teuthology.lock.ops:unlocked: smithi191.front.sepia.ceph.com
2023-03-03T18:18:19.632 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-03T18:18:19.737 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi005 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-03-03_17:39:20-powercycle-reef-distro-default-smithi/7192513
2023-03-03T18:18:19.737 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi191 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-03-03_17:39:20-powercycle-reef-distro-default-smithi/7192513