2021-12-05T19:49:00.674 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-05T19:49:00.798 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi058.front.sepia.ceph.com' reimaging 2021-12-05T19:49:00.799 INFO:teuthology.lock.ops:Updating [ubuntu@smithi058.front.sepia.ceph.com]: reset os type and version on server 2021-12-05T19:49:00.799 INFO:teuthology.lock.ops:Updating smithi058.front.sepia.ceph.com on lock server 2021-12-05T19:49:00.817 INFO:teuthology.lock.ops:Node 'ubuntu@smithi058.front.sepia.ceph.com' reimaging is complete 2021-12-05T19:49:00.818 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi151.front.sepia.ceph.com' reimaging 2021-12-05T19:49:00.818 INFO:teuthology.lock.ops:Updating [ubuntu@smithi151.front.sepia.ceph.com]: reset os type and version on server 2021-12-05T19:49:00.819 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2021-12-05T19:49:00.837 INFO:teuthology.lock.ops:Node 'ubuntu@smithi151.front.sepia.ceph.com' reimaging is complete 2021-12-05T19:49:01.051 INFO:teuthology.provision.fog.smithi058:Scheduling deploy of rhel 8.4 2021-12-05T19:49:01.055 INFO:teuthology.provision.fog.smithi151:Scheduling deploy of rhel 8.4 2021-12-05T19:49:01.400 INFO:teuthology.orchestra.console:Power off smithi058 2021-12-05T19:49:01.415 INFO:teuthology.orchestra.console:Power off smithi151 2021-12-05T19:49:09.781 INFO:teuthology.orchestra.console:Power off for smithi058 completed 2021-12-05T19:49:09.787 INFO:teuthology.orchestra.console:Power off for smithi151 completed 2021-12-05T19:49:09.883 INFO:teuthology.orchestra.console:Power on smithi058 2021-12-05T19:49:09.892 INFO:teuthology.orchestra.console:Power on smithi151 2021-12-05T19:49:14.144 INFO:teuthology.orchestra.console:Power on for smithi151 completed 2021-12-05T19:49:14.245 INFO:teuthology.provision.fog.smithi151:Waiting for deploy to finish 2021-12-05T19:49:18.250 INFO:teuthology.orchestra.console:Power on for smithi058 completed 2021-12-05T19:49:18.352 INFO:teuthology.provision.fog.smithi058:Waiting for deploy to finish 2021-12-05T19:53:52.904 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-05T19:53:53.135 INFO:teuthology.provision.fog.smithi058:Node is ready 2021-12-05T19:53:53.151 INFO:teuthology.orchestra.run.smithi058.stdout:smithi058.front.sepia.ceph.com 2021-12-05T19:53:53.207 INFO:teuthology.orchestra.run.smithi058.stdout:172.21.15.58 smithi058.front.sepia.ceph.com smithi058 2021-12-05T19:53:53.533 INFO:teuthology.provision.fog.smithi058:Deploy complete! 2021-12-05T19:54:15.652 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-05T19:54:15.868 INFO:teuthology.provision.fog.smithi151:Node is ready 2021-12-05T19:54:15.883 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151.front.sepia.ceph.com 2021-12-05T19:54:15.940 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151 smithi151.front.sepia.ceph.com smithi151 2021-12-05T19:54:16.258 INFO:teuthology.provision.fog.smithi151:Deploy complete! 2021-12-05T19:54:16.413 INFO:teuthology.lock.ops:Checking smithi058.front.sepia.ceph.com 2021-12-05T19:54:16.414 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-05T19:54:16.432 INFO:teuthology.lock.ops:Checking smithi151.front.sepia.ceph.com 2021-12-05T19:54:16.432 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-05T19:54:16.449 INFO:teuthology.lock.ops:Updating [smithi058.front.sepia.ceph.com]: set os type and version on server 2021-12-05T19:54:16.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi058.front.sepia.ceph.com' 2021-12-05T19:54:16.692 INFO:teuthology.orchestra.run.smithi058.stdout:x86_64 2021-12-05T19:54:16.749 INFO:teuthology.orchestra.run.smithi058.stdout:NAME="Red Hat Enterprise Linux" 2021-12-05T19:54:16.749 INFO:teuthology.orchestra.run.smithi058.stdout:VERSION="8.4 (Ootpa)" 2021-12-05T19:54:16.750 INFO:teuthology.orchestra.run.smithi058.stdout:ID="rhel" 2021-12-05T19:54:16.750 INFO:teuthology.orchestra.run.smithi058.stdout:ID_LIKE="fedora" 2021-12-05T19:54:16.750 INFO:teuthology.orchestra.run.smithi058.stdout:VERSION_ID="8.4" 2021-12-05T19:54:16.750 INFO:teuthology.orchestra.run.smithi058.stdout:PLATFORM_ID="platform:el8" 2021-12-05T19:54:16.751 INFO:teuthology.orchestra.run.smithi058.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-12-05T19:54:16.751 INFO:teuthology.orchestra.run.smithi058.stdout:ANSI_COLOR="0;31" 2021-12-05T19:54:16.751 INFO:teuthology.orchestra.run.smithi058.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-12-05T19:54:16.751 INFO:teuthology.orchestra.run.smithi058.stdout:HOME_URL="https://www.redhat.com/" 2021-12-05T19:54:16.752 INFO:teuthology.orchestra.run.smithi058.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-12-05T19:54:16.752 INFO:teuthology.orchestra.run.smithi058.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-05T19:54:16.752 INFO:teuthology.orchestra.run.smithi058.stdout: 2021-12-05T19:54:16.752 INFO:teuthology.orchestra.run.smithi058.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-05T19:54:16.752 INFO:teuthology.orchestra.run.smithi058.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-12-05T19:54:16.753 INFO:teuthology.orchestra.run.smithi058.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-12-05T19:54:16.753 INFO:teuthology.orchestra.run.smithi058.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-12-05T19:54:16.754 INFO:teuthology.lock.ops:Updating smithi058.front.sepia.ceph.com on lock server 2021-12-05T19:54:16.772 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: set os type and version on server 2021-12-05T19:54:16.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2021-12-05T19:54:16.996 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2021-12-05T19:54:17.052 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="Red Hat Enterprise Linux" 2021-12-05T19:54:17.053 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8.4 (Ootpa)" 2021-12-05T19:54:17.053 INFO:teuthology.orchestra.run.smithi151.stdout:ID="rhel" 2021-12-05T19:54:17.053 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="fedora" 2021-12-05T19:54:17.053 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8.4" 2021-12-05T19:54:17.054 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8" 2021-12-05T19:54:17.054 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-12-05T19:54:17.054 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2021-12-05T19:54:17.054 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-12-05T19:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://www.redhat.com/" 2021-12-05T19:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-12-05T19:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-05T19:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout: 2021-12-05T19:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-05T19:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-12-05T19:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-12-05T19:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-12-05T19:54:17.057 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2021-12-05T19:54:17.154 INFO:teuthology.dispatcher.supervisor:Running job 6546857 2021-12-05T19:54:17.154 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546857 --name benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} -- /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546857/orig.config.yaml 2021-12-05T19:54:17.158 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546857 2021-12-05T19:54:17.159 INFO:teuthology.dispatcher.supervisor:Job PID: 18670 2021-12-05T19:54:17.159 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-06T07:55:10.820 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-06T07:55:11.008 INFO:teuthology.kill:Killing Pids: {18670} 2021-12-06T07:55:12.234 INFO:teuthology.task.internal:roles: ubuntu@smithi058.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-12-06T07:55:12.235 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-12-06T07:55:12.236 INFO:teuthology.misc:Compressing logs... 2021-12-06T07:55:12.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi058.front.sepia.ceph.com' 2021-12-06T07:55:12.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2021-12-06T07:55:12.838 INFO:teuthology.misc:Compressing logs... 2021-12-06T07:55:13.632 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-06T07:55:13.923 INFO:teuthology.orchestra.run.smithi058.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-06T07:55:14.469 INFO:teuthology.misc:Compressing logs... 2021-12-06T07:55:33.436 INFO:teuthology.kill:No teuthology processes running 2021-12-06T07:55:34.558 INFO:teuthology.kill:Nuking machines: ['smithi058', 'smithi151'] 2021-12-06T07:55:35.497 INFO:teuthology.kill:2021-12-06 07:55:35,497.497 INFO:teuthology.nuke:targets: 2021-12-06T07:55:35.498 INFO:teuthology.kill: smithi058.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe+8OzD16FN7EVST5SuJ1WJo4YcpHK+aLKezJ0L3NGEcnpDvKsbpvz8iAEaXy/SjOYEJs+8LJUdemgtsMnGAcYr//VfuHtIoH+WmFpcWhpVqAljdm6yKkNQzhKBN1sZBr9PodLQ0rxorIhRmSzNlqQcQQ2tBXFlSlOrTAO8upjp+qb5E2jvoVwiXtmwUrKhBDPNiNxRn3whEh9XhkUGxYRUmKtG/e40Ik4HG8OfKIHbk8yNmx2uJYx8UzX9TGu3yBif8tQNIRpE7zFw3NkKCxAKHrRud6w7Ew/mPqi2uC4YihGJZOmJbw06DmABhtnXJnZuXmY5nvPtAdFMKeQQ5ugKjtfnOtc9T7rDH9URWCqz8vao+gmYcVSMeoz82o+972/8NtCBiMIolHekqlnAR7jRksUe8tNg18Fy3Dq2FVUWNqMQSaiQzkfYUkekCrBbR+RWEK1llx2Hs664PxZT4RCrljdjpfQjku6WpHzoQi31DXRHg6GtN8tWop/fZHSaI8= 2021-12-06T07:55:35.498 INFO:teuthology.kill: smithi151.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCx6s1xNNuuEenBydkHxQugDynEenrNEsejzgzE8SskvUe9fJsFt1TorxH6UrQfENRqNAl+UVIm49HcmhGdxP3YaIkC+fNaqZaGdtiPapBFYNfEbxpzcIqDgmre6VixB/ea3BDrpBuqgl2AOAXdfzTw6Mt1oKB5GlpKMOU03xZ/600g5UEXSfhO5RjgQaREmsi/AJ7FzB2iu/gIleDV9U1R4R2pLssUOLUfuzoSFe82W/KJOJvDlCaAfeO57GuTFRak2apItpVoQLR3OTnh0bYlwD7DyNuXGWEoMOZLcLc5CoFtENnue3XLqGE4Z4eOkZm6CdppBpYI4zZKBHtGBgBoeha4WVYLXQr1YJ9SD/pUyasyK+bnc3FXvNm5uVjqT+HVS3O/GBUJ2TRffupS+AasnBM2G9SS54Pxyz8vK/OUlvaoovvUCOf+DrBupF6usYW5DVNJbdWDgR8G05/o632QpSp9Y+YdsnVBDSCSN5bc4i7gPpuNDWK3mZ7+Y7YtSjM= 2021-12-06T07:55:35.525 INFO:teuthology.kill:2021-12-06 07:55:35,525.525 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-06T07:55:35.528 INFO:teuthology.kill:2021-12-06 07:55:35,528.528 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-06T07:55:35.609 INFO:teuthology.kill:2021-12-06 07:55:35,608.608 INFO:teuthology.orchestra.console:Power off smithi058 2021-12-06T07:55:35.622 INFO:teuthology.kill:2021-12-06 07:55:35,622.622 INFO:teuthology.orchestra.console:Power off smithi151 2021-12-06T07:55:43.978 INFO:teuthology.kill:2021-12-06 07:55:43,977.977 INFO:teuthology.orchestra.console:Power off for smithi058 completed 2021-12-06T07:55:43.986 INFO:teuthology.kill:2021-12-06 07:55:43,986.986 INFO:teuthology.orchestra.console:Power off for smithi151 completed 2021-12-06T07:55:44.122 INFO:teuthology.kill:2021-12-06 07:55:44,122.122 INFO:teuthology.lock.ops:unlocked: smithi058.front.sepia.ceph.com 2021-12-06T07:55:44.130 INFO:teuthology.kill:2021-12-06 07:55:44,129.129 INFO:teuthology.lock.ops:unlocked: smithi151.front.sepia.ceph.com 2021-12-06T07:57:44.435 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-06T07:57:44.540 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi058 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_06:07:02-rbd-pacific-distro-default-smithi/6547526 2021-12-06T07:57:44.541 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi151 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_06:07:02-rbd-pacific-distro-default-smithi/6547526