2022-02-07T15:29:02.351 INFO:root:teuthology version: 1.1.0-3094160c 2022-02-07T15:29:02.475 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi018.front.sepia.ceph.com' reimaging 2022-02-07T15:29:02.476 INFO:teuthology.lock.ops:Updating [ubuntu@smithi018.front.sepia.ceph.com]: reset os type and version on server 2022-02-07T15:29:02.477 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2022-02-07T15:29:02.495 INFO:teuthology.lock.ops:Node 'ubuntu@smithi018.front.sepia.ceph.com' reimaging is complete 2022-02-07T15:29:02.495 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging 2022-02-07T15:29:02.496 INFO:teuthology.lock.ops:Updating [ubuntu@smithi059.front.sepia.ceph.com]: reset os type and version on server 2022-02-07T15:29:02.496 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-02-07T15:29:02.514 INFO:teuthology.lock.ops:Node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging is complete 2022-02-07T15:29:02.703 INFO:teuthology.provision.fog.smithi018:Scheduling deploy of rhel 8.4 2022-02-07T15:29:02.714 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of rhel 8.4 2022-02-07T15:29:03.049 INFO:teuthology.orchestra.console:Power off smithi059 2022-02-07T15:29:03.064 INFO:teuthology.orchestra.console:Power off smithi018 2022-02-07T15:29:03.194 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-02-07T15:29:03.205 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2022-02-07T15:29:03.296 INFO:teuthology.orchestra.console:Power on smithi059 2022-02-07T15:29:03.306 INFO:teuthology.orchestra.console:Power on smithi018 2022-02-07T15:29:11.665 INFO:teuthology.orchestra.console:Power on for smithi059 completed 2022-02-07T15:29:11.744 INFO:teuthology.orchestra.console:Power on for smithi018 completed 2022-02-07T15:29:11.767 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2022-02-07T15:29:11.845 INFO:teuthology.provision.fog.smithi018:Waiting for deploy to finish 2022-02-07T15:32:53.871 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-07T15:33:09.461 INFO:teuthology.provision.fog.smithi018:Node is ready 2022-02-07T15:33:09.481 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018.front.sepia.ceph.com 2022-02-07T15:33:09.539 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18 smithi018.front.sepia.ceph.com smithi018 2022-02-07T15:33:09.878 INFO:teuthology.provision.fog.smithi018:Deploy complete! 2022-02-07T15:33:19.324 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-07T15:33:19.569 INFO:teuthology.provision.fog.smithi059:Node is ready 2022-02-07T15:33:19.585 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2022-02-07T15:33:19.641 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2022-02-07T15:33:19.974 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2022-02-07T15:33:20.124 INFO:teuthology.lock.ops:Checking smithi018.front.sepia.ceph.com 2022-02-07T15:33:20.124 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-07T15:33:20.145 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2022-02-07T15:33:20.146 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-07T15:33:20.166 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: set os type and version on server 2022-02-07T15:33:20.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2022-02-07T15:33:20.406 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2022-02-07T15:33:20.461 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="Red Hat Enterprise Linux" 2022-02-07T15:33:20.461 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="8.4 (Ootpa)" 2022-02-07T15:33:20.461 INFO:teuthology.orchestra.run.smithi018.stdout:ID="rhel" 2022-02-07T15:33:20.462 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="fedora" 2022-02-07T15:33:20.462 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="8.4" 2022-02-07T15:33:20.462 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el8" 2022-02-07T15:33:20.462 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-02-07T15:33:20.463 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2022-02-07T15:33:20.463 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-02-07T15:33:20.463 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://www.redhat.com/" 2022-02-07T15:33:20.463 INFO:teuthology.orchestra.run.smithi018.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-02-07T15:33:20.464 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-07T15:33:20.464 INFO:teuthology.orchestra.run.smithi018.stdout: 2022-02-07T15:33:20.464 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-07T15:33:20.464 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-02-07T15:33:20.464 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-02-07T15:33:20.465 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-02-07T15:33:20.466 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2022-02-07T15:33:20.484 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2022-02-07T15:33:20.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-02-07T15:33:20.711 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2022-02-07T15:33:20.765 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2022-02-07T15:33:20.766 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.4 (Ootpa)" 2022-02-07T15:33:20.766 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2022-02-07T15:33:20.766 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2022-02-07T15:33:20.767 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.4" 2022-02-07T15:33:20.767 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2022-02-07T15:33:20.767 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-02-07T15:33:20.767 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2022-02-07T15:33:20.768 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-02-07T15:33:20.768 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2022-02-07T15:33:20.768 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-02-07T15:33:20.768 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-07T15:33:20.769 INFO:teuthology.orchestra.run.smithi059.stdout: 2022-02-07T15:33:20.769 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-07T15:33:20.769 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-02-07T15:33:20.769 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-02-07T15:33:20.770 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-02-07T15:33:20.770 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-02-07T15:33:20.864 INFO:teuthology.dispatcher.supervisor:Running job 6666525 2022-02-07T15:33:20.864 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666525 --name benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} -- /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666525/orig.config.yaml 2022-02-07T15:33:20.868 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666525 2022-02-07T15:33:20.869 INFO:teuthology.dispatcher.supervisor:Job PID: 15797 2022-02-07T15:33:20.869 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-07T22:03:49.971 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-07T22:03:50.218 INFO:teuthology.kill:Killing Pids: {15797} 2022-02-07T22:03:51.395 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-07T22:03:51.396 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-07T22:03:51.396 INFO:teuthology.misc:Compressing logs... 2022-02-07T22:03:51.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2022-02-07T22:03:51.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-02-07T22:03:51.943 INFO:teuthology.misc:Compressing logs... 2022-02-07T22:03:52.493 INFO:teuthology.orchestra.run.smithi018.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-02-07T22:03:52.698 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-02-07T22:03:53.078 INFO:teuthology.misc:Compressing logs... 2022-02-07T22:04:00.265 INFO:teuthology.kill:No teuthology processes running 2022-02-07T22:04:01.374 INFO:teuthology.kill:Nuking machines: ['smithi018', 'smithi059'] 2022-02-07T22:04:02.323 INFO:teuthology.kill:2022-02-07 22:04:02,322.322 INFO:teuthology.nuke:targets: 2022-02-07T22:04:02.324 INFO:teuthology.kill: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4ce4ZK6LxRMZoMQSeZ0yy9kfqz9p33g50kez5jViNSBvaZEXBvHrdZ4p17nPVNSsRNvV78H+mfbVTjg33HEQv0F+/HI6WVRsRKsWFl4mjwORoYUWOtt5RPV4H6lV9a/cKCh+ITkl4F7lSrjYhqswJkXBE5OF5dFjnN8gYAnuFbE6P/Vlvh/SA6kV5CmMc3eZ8sY4D+stgLviqKfnocfQUaUKtYCxfLHB86sNOD/GNe5rDm5VAIc3IXJvTcqU0FKFYGqxcXtcvjn2cb/MXv8W4KnEtGiya+K0a5Olx7Q7I96872buqlxSMpf5fCvfthL340BJu0s6liq2IvwFwVXE0Lbg2Z2sySjmYCKGkq+PwABFT+uaE4DyMZvBKyugdEhANLktQce7btZdlakMg1TC7Gl9ZA2CZe4uX8IEvpaPfjpHp4lTfgiPywUmoItmH6LBK0xuEzWBGJ1CRCII/s2xCIUOxP0psGH9HSg3ozuyldLpPPMyGNsuKIxie3PFFaqE= 2022-02-07T22:04:02.324 INFO:teuthology.kill: smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPkja4sc81s8VqQJ4Vhv/hmPcXrRJ8Eh5KrD4UyC7dxCA3i1/V1KkBKGzVWmQi0IYJMujAbQ9jtDQPDNuwPUeBoJPrULZoQLw8I6ae7PoMFmNs/urfohDQSvxIVzQ6X6s43csBmwLqmt8fVRGg9Ckam+aLIclFVQqMhrgHiw33cA2sBOz0C31SSGiCAawvt5Dzod/D25Age7T40S7PwRzH7NAPThHHhw+I3MSvYOGYZ0A+AEEPrvpMOhaWwWnY3eMKOkE5xeC/m0oBbwR+3jkpekYARYhs+lIpajZ0vUbninL8LGFOOZJsRhOWIFdo+8pe4SkidYxMB8HfKwIpfQm83VFlXHLjdkicxPvOO9pkfkWOGZtR2y0gwQnHVGNEiWhdCgPbgEUFx2eGx/bVhItbw6m/AtqTqA/9BGCXlalT+yw6h2emF1YP7jBBOqQNJJvBQ1iWUFzNIk5UEcrJ11fD+dWYykZJbiPKaKoEjIvkVFQpl+3K1yii2lCvYZbNyFc= 2022-02-07T22:04:02.349 INFO:teuthology.kill:2022-02-07 22:04:02,348.348 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T22:04:02.352 INFO:teuthology.kill:2022-02-07 22:04:02,352.352 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T22:04:02.426 INFO:teuthology.kill:2022-02-07 22:04:02,425.425 INFO:teuthology.orchestra.console:Power off smithi018 2022-02-07T22:04:02.437 INFO:teuthology.kill:2022-02-07 22:04:02,437.437 INFO:teuthology.orchestra.console:Power off smithi059 2022-02-07T22:04:10.793 INFO:teuthology.kill:2022-02-07 22:04:10,792.792 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2022-02-07T22:04:10.806 INFO:teuthology.kill:2022-02-07 22:04:10,806.806 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-02-07T22:04:10.937 INFO:teuthology.kill:2022-02-07 22:04:10,936.936 INFO:teuthology.lock.ops:unlocked: smithi018.front.sepia.ceph.com 2022-02-07T22:04:10.950 INFO:teuthology.kill:2022-02-07 22:04:10,950.950 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2022-02-07T22:06:11.276 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-07T22:06:11.407 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi018 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6667004 2022-02-07T22:06:11.407 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi059 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6667004