2023-03-29T10:12:48.401 INFO:root:teuthology version: 0.0.1.dev78+ge372916
2023-03-29T10:12:48.579 INFO:teuthology.lock.ops:Start node 'smithi100.front.sepia.ceph.com' reimaging
2023-03-29T10:12:48.580 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: reset os type and version on server
2023-03-29T10:12:48.580 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server
2023-03-29T10:12:48.603 INFO:teuthology.lock.ops:Node 'smithi100.front.sepia.ceph.com' reimaging is complete
2023-03-29T10:12:48.604 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging
2023-03-29T10:12:48.604 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server
2023-03-29T10:12:48.604 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2023-03-29T10:12:48.628 INFO:teuthology.lock.ops:Node 'smithi153.front.sepia.ceph.com' reimaging is complete
2023-03-29T10:12:48.772 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of centos 8.stream
2023-03-29T10:12:48.799 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 8.stream
2023-03-29T10:12:49.038 INFO:teuthology.orchestra.console:Power off smithi100
2023-03-29T10:12:49.058 INFO:teuthology.orchestra.console:Power off smithi153
2023-03-29T10:12:57.422 INFO:teuthology.orchestra.console:Power off for smithi100 completed
2023-03-29T10:12:57.523 INFO:teuthology.orchestra.console:Power on smithi100
2023-03-29T10:13:01.548 INFO:teuthology.orchestra.console:Power off for smithi153 completed
2023-03-29T10:13:01.650 INFO:teuthology.orchestra.console:Power on smithi153
2023-03-29T10:13:01.785 INFO:teuthology.orchestra.console:Power on for smithi100 completed
2023-03-29T10:13:01.886 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish
2023-03-29T10:13:05.912 INFO:teuthology.orchestra.console:Power on for smithi153 completed
2023-03-29T10:13:06.013 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish
2023-03-29T10:15:50.757 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.100
2023-03-29T10:15:59.813 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.100
2023-03-29T10:16:08.901 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.100
2023-03-29T10:16:09.861 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-03-29T10:16:17.957 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.100
2023-03-29T10:16:18.917 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-03-29T10:16:27.045 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.100
2023-03-29T10:16:27.973 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-03-29T10:16:36.101 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.100
2023-03-29T10:16:37.033 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-03-29T10:16:46.085 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-03-29T10:16:55.141 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-03-29T10:17:04.197 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-03-29T10:17:41.884 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-29T10:17:42.106 WARNING:teuthology.provision.fog:timed out
2023-03-29T10:17:48.378 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-29T10:17:48.665 INFO:teuthology.provision.fog.smithi100:Node is ready
2023-03-29T10:17:48.683 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com
2023-03-29T10:17:48.742 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100
2023-03-29T10:17:49.097 INFO:teuthology.provision.fog.smithi100:Deploy complete!
2023-03-29T10:18:07.659 INFO:teuthology.provision.fog.smithi153:Node is ready
2023-03-29T10:18:07.682 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com
2023-03-29T10:18:07.741 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153
2023-03-29T10:18:08.117 INFO:teuthology.provision.fog.smithi153:Deploy complete!
2023-03-29T10:18:08.303 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com
2023-03-29T10:18:08.304 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-29T10:18:08.333 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com
2023-03-29T10:18:08.333 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-29T10:18:08.359 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server
2023-03-29T10:18:08.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com'
2023-03-29T10:18:08.613 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64
2023-03-29T10:18:08.673 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream"
2023-03-29T10:18:08.673 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8"
2023-03-29T10:18:08.673 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos"
2023-03-29T10:18:08.673 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora"
2023-03-29T10:18:08.673 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8"
2023-03-29T10:18:08.673 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8"
2023-03-29T10:18:08.674 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-29T10:18:08.674 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31"
2023-03-29T10:18:08.674 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-29T10:18:08.674 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/"
2023-03-29T10:18:08.674 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-29T10:18:08.674 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-29T10:18:08.674 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-29T10:18:08.675 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server
2023-03-29T10:18:08.699 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server
2023-03-29T10:18:08.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2023-03-29T10:18:08.851 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/"
2023-03-29T10:18:08.909 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-29T10:18:08.910 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-29T10:18:08.910 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-29T10:18:08.910 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2023-03-29T10:18:09.023 INFO:teuthology.dispatcher.supervisor:Running job 7224504
2023-03-29T10:18:09.023 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2023-03-28_22:46:34-rados-WIP_GBH_snap_mapper_B-distro-default-smithi/7224504 --name benhanokh-2023-03-28_22:46:34-rados-WIP_GBH_snap_mapper_B-distro-default-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2023-03-28_22:46:34-rados-WIP_GBH_snap_mapper_B-distro-default-smithi/7224504/orig.config.yaml
2023-03-29T10:18:09.027 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2023-03-28_22:46:34-rados-WIP_GBH_snap_mapper_B-distro-default-smithi/7224504
2023-03-29T10:18:09.028 INFO:teuthology.dispatcher.supervisor:Job PID: 17091
2023-03-29T10:18:09.028 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-29T22:19:07.049 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-29T22:19:07.229 INFO:teuthology.kill:Killing Pids: {17091}
2023-03-29T22:19:08.206 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2023-03-29T22:19:08.206 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2023-03-29T22:19:08.206 INFO:teuthology.misc:Compressing logs...
2023-03-29T22:19:08.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com'
2023-03-29T22:19:08.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2023-03-29T22:19:08.805 INFO:teuthology.misc:Compressing logs...
2023-03-29T22:19:09.575 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-29T22:19:09.832 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-29T22:19:43.623 INFO:teuthology.misc:Compressing logs...
2023-03-29T22:19:46.348 INFO:teuthology.kill:No teuthology processes running
2023-03-29T22:19:47.216 INFO:teuthology.kill:Nuking machines: ['smithi100', 'smithi153']
2023-03-29T22:19:47.905 INFO:teuthology.kill:2023-03-29 22:19:47,905.905 INFO:teuthology.nuke:targets:
2023-03-29T22:19:47.906 INFO:teuthology.kill:  smithi100.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGrNx2obFCM65QDURBhAGuyJdcX6f606jEXbjbhLEvkpKtGl5GbqPMIeBOQnU2PwYgbYckdgGHaWaXx7v5ySJYA1ZyaxyzvRZ2hOr/kWl0v4LlWJuUQB3+8Q5M6DN+q0SeHDxdQPwpcb3oBPxDrItthWAyGcg0nPq3o8X5ZrhKQ0D23OeuFIfGcVyeqTOflcWAxmdGKznmaKon3lKMPqAbr27/dmHBgILDwvhAQC6+E8kmBWWONwLM+Jvfvp1kND8Y1t2rFRcF50eKSrlMyhjLwTkRrfpmsIBtOA1oBQu7LhHZIxs9mDGMie0Hof33z+Ih5xP1ttrecKEC7gxCj4f8nfS/YORogjB7t6fOAyJMyeM6XoWM+Qi3QbfiqLUPxqd90ReShcN2FirVnQ5CzMJ29ArpTpEm0fHoCTirH54jRLff1keXxPskc2rpjuYRVYm0mQ/+ENiOpZA/t5zDNo5lMndbZsDMAU8jCDnbtSvTpk6O9xLpbliP2P6f7ocDuLs=
2023-03-29T22:19:47.906 INFO:teuthology.kill:  smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbdBL8z/uNj7YYuoegCMI5bwmyGfqtUPobcbKorUrglxpw1JKfrewbclT1iD+vC96poWJsOXExaoMx5NFUSMPmQe9VpFDJC94K2HFodqAhZ8+0ogaISmSULsKIZ1jOHdMDx8ISSJwf+7dC9Lx6AFkJYKik3+mu19345rU5wkiNaYoA2rinIlWDqEwpbRkCh+9x+45dU10gCfbFYd30jo395XkyagPClH+bxPvqGjygNikx9vkkH+T+LxYqplm5tH7oOHofkAV+/0HThPsQXCvtDvOE8gVYg2rAOYciUrVj1/8mlUgr+gW4nh/srZR1e/Pe4sWUHuJKgURMWY/qDTRAaLGYP/3cz4mpt7zdDNoT8Dab/rkqQlSKZkFw3Xa33tjCOQ1hziZ2lARHoHeCRPb/zpM6opm+qtMCKq1HReoWdO7k8vexT0pvT+z97OIu/dA8ZBZKVrHqfcfD9ZCzrfIzyZTb5zJRYot6uzj1tRTHAX6A00T28VIcOZJTaG3z6ik=
2023-03-29T22:19:47.906 INFO:teuthology.kill:2023-03-29 22:19:47,905.905 INFO:teuthology.nuke:Checking targets against current locks
2023-03-29T22:19:47.966 INFO:teuthology.kill:2023-03-29 22:19:47,966.966 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-29T22:19:47.992 INFO:teuthology.kill:2023-03-29 22:19:47,992.992 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-29T22:19:48.054 INFO:teuthology.kill:2023-03-29 22:19:48,053.053 INFO:teuthology.orchestra.console:Power off smithi100
2023-03-29T22:19:48.093 INFO:teuthology.kill:2023-03-29 22:19:48,093.093 INFO:teuthology.orchestra.console:Power off smithi153
2023-03-29T22:20:00.546 INFO:teuthology.kill:2023-03-29 22:20:00,545.545 INFO:teuthology.orchestra.console:Power off for smithi100 completed
2023-03-29T22:20:00.579 INFO:teuthology.kill:2023-03-29 22:20:00,579.579 INFO:teuthology.orchestra.console:Power off for smithi153 completed
2023-03-29T22:20:00.706 INFO:teuthology.kill:2023-03-29 22:20:00,705.705 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com
2023-03-29T22:20:00.728 INFO:teuthology.kill:2023-03-29 22:20:00,728.728 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com
2023-03-29T22:22:01.103 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-29T22:22:01.211 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi100 but it was locked by another job: /home/teuthworker/archive/cbodley-2023-03-29_20:47:34-rgw-wip-cbodley-testing-distro-default-smithi/7225925
2023-03-29T22:22:01.211 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/cbodley-2023-03-29_20:47:34-rgw-wip-cbodley-testing-distro-default-smithi/7225926