2021-12-06T23:10:01.091 INFO:root:teuthology version: 1.1.0-844d1377
2021-12-06T23:10:01.220 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi040.front.sepia.ceph.com' reimaging
2021-12-06T23:10:01.222 INFO:teuthology.lock.ops:Updating [ubuntu@smithi040.front.sepia.ceph.com]: reset os type and version on server
2021-12-06T23:10:01.222 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2021-12-06T23:10:01.268 INFO:teuthology.lock.ops:Node 'ubuntu@smithi040.front.sepia.ceph.com' reimaging is complete
2021-12-06T23:10:01.268 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi186.front.sepia.ceph.com' reimaging
2021-12-06T23:10:01.268 INFO:teuthology.lock.ops:Updating [ubuntu@smithi186.front.sepia.ceph.com]: reset os type and version on server
2021-12-06T23:10:01.269 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server
2021-12-06T23:10:01.310 INFO:teuthology.lock.ops:Node 'ubuntu@smithi186.front.sepia.ceph.com' reimaging is complete
2021-12-06T23:10:01.474 INFO:teuthology.provision.fog.smithi040:Scheduling deploy of rhel 8.4
2021-12-06T23:10:01.516 INFO:teuthology.provision.fog.smithi186:Scheduling deploy of rhel 8.4
2021-12-06T23:10:01.890 INFO:teuthology.orchestra.console:Power off smithi186
2021-12-06T23:10:02.031 INFO:teuthology.orchestra.console:Power off for smithi186 completed
2021-12-06T23:10:02.134 INFO:teuthology.orchestra.console:Power on smithi186
2021-12-06T23:10:02.169 INFO:teuthology.orchestra.console:Power off smithi040
2021-12-06T23:10:02.313 INFO:teuthology.orchestra.console:Power off for smithi040 completed
2021-12-06T23:10:02.414 INFO:teuthology.orchestra.console:Power on smithi040
2021-12-06T23:10:06.388 INFO:teuthology.orchestra.console:Power on for smithi186 completed
2021-12-06T23:10:06.490 INFO:teuthology.provision.fog.smithi186:Waiting for deploy to finish
2021-12-06T23:10:10.780 INFO:teuthology.orchestra.console:Power on for smithi040 completed
2021-12-06T23:10:10.881 INFO:teuthology.provision.fog.smithi040:Waiting for deploy to finish
2021-12-06T23:14:48.646 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-06T23:14:48.882 INFO:teuthology.provision.fog.smithi040:Node is ready
2021-12-06T23:14:48.901 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040.front.sepia.ceph.com
2021-12-06T23:14:48.959 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40 smithi040.front.sepia.ceph.com smithi040
2021-12-06T23:14:49.307 INFO:teuthology.provision.fog.smithi040:Deploy complete!
2021-12-06T23:15:11.824 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-06T23:15:12.037 INFO:teuthology.provision.fog.smithi186:Node is ready
2021-12-06T23:15:12.053 INFO:teuthology.orchestra.run.smithi186.stdout:smithi186.front.sepia.ceph.com
2021-12-06T23:15:12.108 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.15.186 smithi186.front.sepia.ceph.com smithi186
2021-12-06T23:15:12.423 INFO:teuthology.provision.fog.smithi186:Deploy complete!
2021-12-06T23:15:12.580 INFO:teuthology.lock.ops:Checking smithi040.front.sepia.ceph.com
2021-12-06T23:15:12.581 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-06T23:15:12.606 INFO:teuthology.lock.ops:Checking smithi186.front.sepia.ceph.com
2021-12-06T23:15:12.606 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-06T23:15:12.629 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: set os type and version on server
2021-12-06T23:15:12.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com'
2021-12-06T23:15:12.881 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64
2021-12-06T23:15:12.937 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Red Hat Enterprise Linux"
2021-12-06T23:15:12.937 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="8.4 (Ootpa)"
2021-12-06T23:15:12.938 INFO:teuthology.orchestra.run.smithi040.stdout:ID="rhel"
2021-12-06T23:15:12.938 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="fedora"
2021-12-06T23:15:12.938 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="8.4"
2021-12-06T23:15:12.938 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el8"
2021-12-06T23:15:12.938 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2021-12-06T23:15:12.939 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31"
2021-12-06T23:15:12.939 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2021-12-06T23:15:12.939 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.redhat.com/"
2021-12-06T23:15:12.939 INFO:teuthology.orchestra.run.smithi040.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2021-12-06T23:15:12.940 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-06T23:15:12.940 INFO:teuthology.orchestra.run.smithi040.stdout:
2021-12-06T23:15:12.940 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-06T23:15:12.940 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2021-12-06T23:15:12.940 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-12-06T23:15:12.942 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2021-12-06T23:15:12.943 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2021-12-06T23:15:12.963 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: set os type and version on server
2021-12-06T23:15:12.964 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com'
2021-12-06T23:15:13.193 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64
2021-12-06T23:15:13.248 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux"
2021-12-06T23:15:13.249 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.4 (Ootpa)"
2021-12-06T23:15:13.249 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel"
2021-12-06T23:15:13.249 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora"
2021-12-06T23:15:13.249 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.4"
2021-12-06T23:15:13.250 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8"
2021-12-06T23:15:13.250 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2021-12-06T23:15:13.250 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31"
2021-12-06T23:15:13.250 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2021-12-06T23:15:13.251 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/"
2021-12-06T23:15:13.251 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2021-12-06T23:15:13.251 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-06T23:15:13.251 INFO:teuthology.orchestra.run.smithi186.stdout:
2021-12-06T23:15:13.252 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-06T23:15:13.252 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2021-12-06T23:15:13.252 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-12-06T23:15:13.252 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2021-12-06T23:15:13.253 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server
2021-12-06T23:15:13.382 INFO:teuthology.dispatcher.supervisor:Running job 6547844
2021-12-06T23:15:13.383 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547844 --name benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi --description rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} -- /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547844/orig.config.yaml
2021-12-06T23:15:13.386 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547844
2021-12-06T23:15:13.387 INFO:teuthology.dispatcher.supervisor:Job PID: 23590
2021-12-06T23:15:13.387 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-07T11:15:33.587 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-07T11:15:33.775 INFO:teuthology.kill:Killing Pids: {23590}
2021-12-07T11:15:35.018 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a']
2021-12-07T11:15:35.019 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2021-12-07T11:15:35.019 INFO:teuthology.misc:Compressing logs...
2021-12-07T11:15:35.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com'
2021-12-07T11:15:35.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com'
2021-12-07T11:15:36.026 INFO:teuthology.misc:Compressing logs...
2021-12-07T11:15:36.993 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-07T11:15:37.319 INFO:teuthology.misc:Compressing logs...
2021-12-07T11:17:27.732 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-07T11:17:41.092 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2021-12-07T11:17:48.132 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-07T11:18:25.632 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2021-12-07T11:18:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2021-12-07T11:19:38.563 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-07T11:20:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-07T11:21:44.295 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-07T11:23:09.108 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-07T11:24:52.043 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-07T11:25:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2021-12-07T11:27:26.023 INFO:teuthology.orchestra.run.smithi186.stderr:tar: ./5d7f069a-56ed-11ec-8c2e-001a4aab830c/ceph-osd.7.log.gz: file changed as we read it
2021-12-07T11:28:15.595 INFO:teuthology.orchestra.run.smithi186.stderr:tar: ./5d7f069a-56ed-11ec-8c2e-001a4aab830c: file changed as we read it
2021-12-07T11:28:15.677 INFO:teuthology.kill:No teuthology processes running
2021-12-07T11:28:16.805 INFO:teuthology.kill:Nuking machines: ['smithi040', 'smithi186']
2021-12-07T11:28:17.789 INFO:teuthology.kill:2021-12-07 11:28:17,788.788 INFO:teuthology.nuke:targets:
2021-12-07T11:28:17.790 INFO:teuthology.kill:  smithi040.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPLWziIDxIa64JuN4XjPnMscYkzT16BXqsvCq6RnmolSTZTVh68FW84TuoZqYYMDiMsynhL/JqwXeEwdniV4Cj91K3yyjMR5iRi9BvnczSrokm8EnlYQkAglWbvEEfcqdKVHCiUHtJbEz1cDp04cvBCN6k40XeXnBkMZvHhertHqkDby+dEJdXfg8nl2ITYcTk3nnURnrql9LedYs+qnA75r69DcD0pQkhT2fwBr9TkkojICBvJI8n6X9DwGgQpjVeEDGx2NV+w/qtdc505nOB4W55xSl9N28JRrB6b/anxkf6mPbHg+F+P5zIJSTUfFRjQOMesyFSzyQw7B4xznvsoB8wKbYiRpYil7rcpH/bX4fOIp0fYeZQvoYoXrZqoAnXiETjtcqXT++ysAxfikjr6NDG+Xq7Haqk23Fnl/xlz+/mFrnkaburJ8/L6KYLgxoQ0dTK1AmqBUb+jas208h9YnEdZQFbHt0lM70EMdazWpVK5LA2XqjGEksDLzjEdOU=
2021-12-07T11:28:17.790 INFO:teuthology.kill:  smithi186.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZj7cTH8N383KATklQcIMEc2ZQPYmaxWYJF317dSGQbzU2e2VOC0SHndUvmitsT1SofwnONZAZ99lALxiP+pvg7qzjbaHcePB57TSDnJtZv38kGbAqw5vLxF9XWMrza5qHARKyRHsNddtNW189yhhzQFhK61U39hzZ+yP1ffdAHNFE8a0FQnXg5bWoO1UOJ7bZH7Vlt2vceSyN39Px4D9YnkaCS3hTc/P++iQUSa5xmvENejtOSkXbxLbgmQwY11QPYqrrVO8zhEGadzvzNURYkg334Lk3wL1zWk4hrX/9jdu+tXyTO0y0LVwPz9MJzbeEQEfa2LBHaaZzGsqSPCzzjEclIidKVjAP84GytgYMlkhdOBycfkM5nJqHyKiV6xbFWwdMypqxLa3CW2YMPHFHPB5W2752I6LpblUArToA2A/4boCd2K/7rXWbppfkZv3MWun6Y9kxGKXH9qVDM4EQdrwWsXgqiQgf2q/ycAsrnaE/FBvRJ1cKsrhazIeCCUc=
2021-12-07T11:28:17.820 INFO:teuthology.kill:2021-12-07 11:28:17,820.820 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-07T11:28:17.825 INFO:teuthology.kill:2021-12-07 11:28:17,825.825 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-07T11:28:17.899 INFO:teuthology.kill:2021-12-07 11:28:17,899.899 INFO:teuthology.orchestra.console:Power off smithi040
2021-12-07T11:28:17.912 INFO:teuthology.kill:2021-12-07 11:28:17,912.912 INFO:teuthology.orchestra.console:Power off smithi186
2021-12-07T11:28:26.271 INFO:teuthology.kill:2021-12-07 11:28:26,271.271 INFO:teuthology.orchestra.console:Power off for smithi040 completed
2021-12-07T11:28:26.277 INFO:teuthology.kill:2021-12-07 11:28:26,276.276 INFO:teuthology.orchestra.console:Power off for smithi186 completed
2021-12-07T11:28:26.419 INFO:teuthology.kill:2021-12-07 11:28:26,418.418 INFO:teuthology.lock.ops:unlocked: smithi040.front.sepia.ceph.com
2021-12-07T11:28:26.422 INFO:teuthology.kill:2021-12-07 11:28:26,422.422 INFO:teuthology.lock.ops:unlocked: smithi186.front.sepia.ceph.com
2021-12-07T11:30:26.740 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-07T11:30:26.834 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi040 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549065
2021-12-07T11:30:26.835 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi186 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549065