2023-03-25T06:44:36.036 INFO:root:teuthology version: 0.0.1.dev1+ge372916
2023-03-25T06:44:36.170 INFO:teuthology.lock.ops:Start node 'smithi070.front.sepia.ceph.com' reimaging
2023-03-25T06:44:36.172 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: reset os type and version on server
2023-03-25T06:44:36.172 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2023-03-25T06:44:36.195 INFO:teuthology.lock.ops:Node 'smithi070.front.sepia.ceph.com' reimaging is complete
2023-03-25T06:44:36.195 INFO:teuthology.lock.ops:Start node 'smithi139.front.sepia.ceph.com' reimaging
2023-03-25T06:44:36.196 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: reset os type and version on server
2023-03-25T06:44:36.196 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server
2023-03-25T06:44:36.220 INFO:teuthology.lock.ops:Node 'smithi139.front.sepia.ceph.com' reimaging is complete
2023-03-25T06:44:36.365 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of rhel 8.6
2023-03-25T06:44:36.371 INFO:teuthology.provision.fog.smithi139:Scheduling deploy of rhel 8.6
2023-03-25T06:44:36.569 INFO:teuthology.orchestra.console:Power off smithi070
2023-03-25T06:44:36.584 INFO:teuthology.orchestra.console:Power off smithi139
2023-03-25T06:44:36.714 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2023-03-25T06:44:36.815 INFO:teuthology.orchestra.console:Power on smithi070
2023-03-25T06:44:44.954 INFO:teuthology.orchestra.console:Power off for smithi139 completed
2023-03-25T06:44:45.055 INFO:teuthology.orchestra.console:Power on smithi139
2023-03-25T06:44:45.186 INFO:teuthology.orchestra.console:Power on for smithi070 completed
2023-03-25T06:44:45.287 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish
2023-03-25T06:44:49.311 INFO:teuthology.orchestra.console:Power on for smithi139 completed
2023-03-25T06:44:49.412 INFO:teuthology.provision.fog.smithi139:Waiting for deploy to finish
2023-03-25T06:47:34.053 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-03-25T06:47:38.117 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.139
2023-03-25T06:47:43.109 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-03-25T06:47:47.173 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.139
2023-03-25T06:47:52.165 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-03-25T06:47:56.229 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.139
2023-03-25T06:48:01.225 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-03-25T06:48:05.285 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.139
2023-03-25T06:48:10.277 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-03-25T06:48:14.341 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.139
2023-03-25T06:48:19.333 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-03-25T06:48:23.397 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.139
2023-03-25T06:49:25.381 WARNING:teuthology.provision.fog:timed out
2023-03-25T06:49:29.401 WARNING:teuthology.provision.fog:timed out
2023-03-25T06:49:31.631 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-25T06:49:31.907 INFO:teuthology.provision.fog.smithi070:Node is ready
2023-03-25T06:49:31.923 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070.front.sepia.ceph.com
2023-03-25T06:49:31.983 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70 smithi070.front.sepia.ceph.com smithi070
2023-03-25T06:49:32.332 INFO:teuthology.provision.fog.smithi070:Deploy complete!
2023-03-25T06:49:35.738 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-25T06:49:36.059 INFO:teuthology.provision.fog.smithi139:Node is ready
2023-03-25T06:49:36.077 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139.front.sepia.ceph.com
2023-03-25T06:49:36.134 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139 smithi139.front.sepia.ceph.com smithi139
2023-03-25T06:49:36.470 INFO:teuthology.provision.fog.smithi139:Deploy complete!
2023-03-25T06:49:36.669 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com
2023-03-25T06:49:36.671 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-25T06:49:36.697 INFO:teuthology.lock.ops:Checking smithi139.front.sepia.ceph.com
2023-03-25T06:49:36.698 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-25T06:49:36.724 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server
2023-03-25T06:49:36.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com'
2023-03-25T06:49:36.918 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64
2023-03-25T06:49:36.976 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Red Hat Enterprise Linux"
2023-03-25T06:49:36.976 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8.6 (Ootpa)"
2023-03-25T06:49:36.977 INFO:teuthology.orchestra.run.smithi070.stdout:ID="rhel"
2023-03-25T06:49:36.977 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="fedora"
2023-03-25T06:49:36.977 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8.6"
2023-03-25T06:49:36.977 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8"
2023-03-25T06:49:36.977 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-25T06:49:36.977 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31"
2023-03-25T06:49:36.978 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-25T06:49:36.978 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.redhat.com/"
2023-03-25T06:49:36.978 INFO:teuthology.orchestra.run.smithi070.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-25T06:49:36.978 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-25T06:49:36.978 INFO:teuthology.orchestra.run.smithi070.stdout:
2023-03-25T06:49:36.978 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-25T06:49:36.978 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-25T06:49:36.979 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-25T06:49:36.979 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-25T06:49:36.980 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2023-03-25T06:49:36.998 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: set os type and version on server
2023-03-25T06:49:36.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com'
2023-03-25T06:49:37.181 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64
2023-03-25T06:49:37.234 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="Red Hat Enterprise Linux"
2023-03-25T06:49:37.234 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="8.6 (Ootpa)"
2023-03-25T06:49:37.234 INFO:teuthology.orchestra.run.smithi139.stdout:ID="rhel"
2023-03-25T06:49:37.234 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="fedora"
2023-03-25T06:49:37.235 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="8.6"
2023-03-25T06:49:37.235 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el8"
2023-03-25T06:49:37.235 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-25T06:49:37.235 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31"
2023-03-25T06:49:37.235 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-25T06:49:37.235 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://www.redhat.com/"
2023-03-25T06:49:37.235 INFO:teuthology.orchestra.run.smithi139.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-25T06:49:37.236 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-25T06:49:37.236 INFO:teuthology.orchestra.run.smithi139.stdout:
2023-03-25T06:49:37.236 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-25T06:49:37.236 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-25T06:49:37.236 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-25T06:49:37.236 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-25T06:49:37.237 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server
2023-03-25T06:49:37.355 INFO:teuthology.dispatcher.supervisor:Running job 7210994
2023-03-25T06:49:37.355 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_main/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2023-03-16_20:57:19-rados-WIP_GBH_snap_mapper_5-distro-default-smithi/7210994 --name benhanokh-2023-03-16_20:57:19-rados-WIP_GBH_snap_mapper_5-distro-default-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2023-03-16_20:57:19-rados-WIP_GBH_snap_mapper_5-distro-default-smithi/7210994/orig.config.yaml
2023-03-25T06:49:37.359 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2023-03-16_20:57:19-rados-WIP_GBH_snap_mapper_5-distro-default-smithi/7210994
2023-03-25T06:49:37.360 INFO:teuthology.dispatcher.supervisor:Job PID: 25893
2023-03-25T06:49:37.360 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-25T18:50:34.961 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-25T18:50:35.120 INFO:teuthology.kill:Killing Pids: {25893}
2023-03-25T18:50:35.969 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2023-03-25T18:50:35.970 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2023-03-25T18:50:35.970 INFO:teuthology.misc:Compressing logs...
2023-03-25T18:50:35.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com'
2023-03-25T18:50:36.154 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com'
2023-03-25T18:50:36.634 INFO:teuthology.misc:Compressing logs...
2023-03-25T18:50:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-25T18:50:37.746 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-25T18:51:56.010 INFO:teuthology.misc:Compressing logs...
2023-03-25T18:51:57.888 INFO:teuthology.kill:No teuthology processes running
2023-03-25T18:51:58.776 INFO:teuthology.kill:Nuking machines: ['smithi070', 'smithi139']
2023-03-25T18:51:59.403 INFO:teuthology.kill:2023-03-25 18:51:59,403.403 INFO:teuthology.nuke:targets:
2023-03-25T18:51:59.404 INFO:teuthology.kill:  smithi070.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-03-25T18:51:59.404 INFO:teuthology.kill:  smithi139.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-03-25T18:51:59.405 INFO:teuthology.kill:2023-03-25 18:51:59,403.403 INFO:teuthology.nuke:Checking targets against current locks
2023-03-25T18:51:59.460 INFO:teuthology.kill:2023-03-25 18:51:59,460.460 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-25T18:51:59.478 INFO:teuthology.kill:2023-03-25 18:51:59,478.478 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-25T18:51:59.547 INFO:teuthology.kill:2023-03-25 18:51:59,546.546 INFO:teuthology.orchestra.console:Power off smithi139
2023-03-25T18:51:59.584 INFO:teuthology.kill:2023-03-25 18:51:59,583.583 INFO:teuthology.orchestra.console:Power off smithi070
2023-03-25T18:52:07.952 INFO:teuthology.kill:2023-03-25 18:52:07,952.952 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2023-03-25T18:52:08.117 INFO:teuthology.kill:2023-03-25 18:52:08,116.116 INFO:teuthology.lock.ops:unlocked: smithi070.front.sepia.ceph.com
2023-03-25T18:52:12.035 INFO:teuthology.kill:2023-03-25 18:52:12,035.035 INFO:teuthology.orchestra.console:Power off for smithi139 completed
2023-03-25T18:52:12.200 INFO:teuthology.kill:2023-03-25 18:52:12,200.200 INFO:teuthology.lock.ops:unlocked: smithi139.front.sepia.ceph.com
2023-03-25T18:54:12.575 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-25T18:54:12.681 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi070 but it was locked by another job: /home/teuthworker/archive/vshankar-2023-03-18_05:01:47-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi/7213544
2023-03-25T18:54:12.681 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi139 but it was locked by another job: /home/teuthworker/archive/vshankar-2023-03-18_05:01:47-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi/7213544