2022-01-20T13:56:11.348 INFO:root:teuthology version: 1.1.0-3ff86a5d 2022-01-20T13:56:11.468 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging 2022-01-20T13:56:11.469 INFO:teuthology.lock.ops:Updating [ubuntu@smithi121.front.sepia.ceph.com]: reset os type and version on server 2022-01-20T13:56:11.470 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-01-20T13:56:11.489 INFO:teuthology.lock.ops:Node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging is complete 2022-01-20T13:56:11.490 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi135.front.sepia.ceph.com' reimaging 2022-01-20T13:56:11.490 INFO:teuthology.lock.ops:Updating [ubuntu@smithi135.front.sepia.ceph.com]: reset os type and version on server 2022-01-20T13:56:11.490 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2022-01-20T13:56:11.509 INFO:teuthology.lock.ops:Node 'ubuntu@smithi135.front.sepia.ceph.com' reimaging is complete 2022-01-20T13:56:11.708 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of rhel 8.4 2022-01-20T13:56:11.748 INFO:teuthology.provision.fog.smithi135:Scheduling deploy of rhel 8.4 2022-01-20T13:56:12.121 INFO:teuthology.orchestra.console:Power off smithi121 2022-01-20T13:56:12.185 INFO:teuthology.orchestra.console:Power off smithi135 2022-01-20T13:56:24.599 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-01-20T13:56:24.660 INFO:teuthology.orchestra.console:Power off for smithi135 completed 2022-01-20T13:56:24.701 INFO:teuthology.orchestra.console:Power on smithi121 2022-01-20T13:56:24.761 INFO:teuthology.orchestra.console:Power on smithi135 2022-01-20T13:56:28.957 INFO:teuthology.orchestra.console:Power on for smithi121 completed 2022-01-20T13:56:29.058 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2022-01-20T13:56:33.162 INFO:teuthology.orchestra.console:Power on for smithi135 completed 2022-01-20T13:56:33.263 INFO:teuthology.provision.fog.smithi135:Waiting for deploy to finish 2022-01-20T14:01:05.058 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-20T14:01:05.268 INFO:teuthology.provision.fog.smithi135:Node is ready 2022-01-20T14:01:05.284 INFO:teuthology.orchestra.run.smithi135.stdout:smithi135.front.sepia.ceph.com 2022-01-20T14:01:05.341 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135 smithi135.front.sepia.ceph.com smithi135 2022-01-20T14:01:05.666 INFO:teuthology.provision.fog.smithi135:Deploy complete! 2022-01-20T14:01:16.170 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-20T14:01:16.363 INFO:teuthology.provision.fog.smithi121:Node is ready 2022-01-20T14:01:16.418 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com 2022-01-20T14:01:16.473 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121 2022-01-20T14:01:16.794 INFO:teuthology.provision.fog.smithi121:Deploy complete! 2022-01-20T14:01:16.949 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com 2022-01-20T14:01:16.950 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-20T14:01:16.969 INFO:teuthology.lock.ops:Checking smithi135.front.sepia.ceph.com 2022-01-20T14:01:16.970 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-20T14:01:16.993 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server 2022-01-20T14:01:16.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-01-20T14:01:17.215 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2022-01-20T14:01:17.270 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux" 2022-01-20T14:01:17.271 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.4 (Ootpa)" 2022-01-20T14:01:17.271 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel" 2022-01-20T14:01:17.271 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora" 2022-01-20T14:01:17.271 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.4" 2022-01-20T14:01:17.272 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2022-01-20T14:01:17.272 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-20T14:01:17.272 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2022-01-20T14:01:17.272 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-20T14:01:17.272 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/" 2022-01-20T14:01:17.273 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-20T14:01:17.273 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-20T14:01:17.273 INFO:teuthology.orchestra.run.smithi121.stdout: 2022-01-20T14:01:17.273 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-20T14:01:17.274 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-20T14:01:17.274 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-20T14:01:17.274 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-20T14:01:17.276 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-01-20T14:01:17.294 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: set os type and version on server 2022-01-20T14:01:17.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2022-01-20T14:01:17.530 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2022-01-20T14:01:17.586 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Red Hat Enterprise Linux" 2022-01-20T14:01:17.586 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="8.4 (Ootpa)" 2022-01-20T14:01:17.587 INFO:teuthology.orchestra.run.smithi135.stdout:ID="rhel" 2022-01-20T14:01:17.587 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="fedora" 2022-01-20T14:01:17.587 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="8.4" 2022-01-20T14:01:17.587 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el8" 2022-01-20T14:01:17.588 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-20T14:01:17.588 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2022-01-20T14:01:17.588 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-20T14:01:17.588 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.redhat.com/" 2022-01-20T14:01:17.589 INFO:teuthology.orchestra.run.smithi135.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-20T14:01:17.589 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-20T14:01:17.589 INFO:teuthology.orchestra.run.smithi135.stdout: 2022-01-20T14:01:17.589 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-20T14:01:17.589 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-20T14:01:17.590 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-20T14:01:17.590 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-20T14:01:17.590 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2022-01-20T14:01:17.696 INFO:teuthology.dispatcher.supervisor:Running job 6624095 2022-01-20T14:01:17.697 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3ff86a5dbaafecdb6414fa2fea970bb44e836829/virtualenv/bin/teuthology -v --owner scheduled_ifed01@teuthology --archive /home/teuthworker/archive/ifed01-2022-01-18_11:04:04-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6624095 --name ifed01-2022-01-18_11:04:04-rados-wip-ifed-faster-rm-p2-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/ifed01-2022-01-18_11:04:04-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6624095/orig.config.yaml 2022-01-20T14:01:17.700 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ifed01-2022-01-18_11:04:04-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6624095 2022-01-20T14:01:17.701 INFO:teuthology.dispatcher.supervisor:Job PID: 4833 2022-01-20T14:01:17.702 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-20T20:31:47.289 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-20T20:31:47.495 INFO:teuthology.kill:Killing Pids: {4833} 2022-01-20T20:31:49.379 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-20T20:31:49.381 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-20T20:31:49.381 INFO:teuthology.misc:Compressing logs... 2022-01-20T20:31:49.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-01-20T20:31:49.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2022-01-20T20:31:49.622 INFO:teuthology.orchestra.run.smithi121.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-20T20:31:49.856 INFO:teuthology.orchestra.run.smithi135.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-20T20:31:49.888 INFO:teuthology.orchestra.run.smithi121.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-20T20:31:49.888 INFO:teuthology.orchestra.run.smithi121.stderr:tar: Error is not recoverable: exiting now 2022-01-20T20:31:49.915 INFO:teuthology.orchestra.run.smithi135.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-20T20:31:49.915 INFO:teuthology.orchestra.run.smithi135.stderr:tar: Error is not recoverable: exiting now 2022-01-20T20:31:49.916 INFO:teuthology.misc:Compressing logs... 2022-01-20T20:31:50.115 INFO:teuthology.misc:Compressing logs... 2022-01-20T20:31:59.320 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-mon.smithi121.log: file size changed while zipping 2022-01-20T20:31:59.542 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-20T20:32:03.020 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-mon.smithi135.log: file size changed while zipping 2022-01-20T20:32:06.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-20T20:32:08.297 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-mgr.smithi121.uwmzfv.log: file size changed while zipping 2022-01-20T20:32:09.952 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-20T20:32:15.616 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-20T20:32:19.246 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-20T20:32:23.706 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-20T20:32:26.470 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-20T20:32:33.457 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/e676e532-79fa-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-20T20:32:50.315 INFO:teuthology.kill:No teuthology processes running 2022-01-20T20:32:51.618 INFO:teuthology.kill:Nuking machines: ['smithi121', 'smithi135'] 2022-01-20T20:32:52.666 INFO:teuthology.kill:2022-01-20 20:32:52,666.666 INFO:teuthology.nuke:targets: 2022-01-20T20:32:52.668 INFO:teuthology.kill: smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCitwMWdhZ7YHKsIC7XyEsIA3BvfsDZOmBG7BWWbaRXgec8d7yt2WMz3ZP/z/qZnosHTty8G9pjChsd0PDN6qFLu7GjecbhESgZGXBW/LEt6xT/bycKJPtEwQ4QWQDisq/2BAKL/tgw/cavSt602WtKoRXVG+Owry8rQ+iDwNJtdN+EZTRPpaC288jR9UpwvHZcwm/EuQ4x8cTbngE/UB0YyN0Kafi9UDMQelE40tvDEKC1uQ8c0OzhXvucrdMjpMbCIvgJCY/HrAJA3tyBucUsasj3PJF113m0lOMdtlRU2tdloSY1NO8dzoMapVeevdQ7PnxRRRAxF5Ut+RZLlC1n5GZM+2R/6M3sMsQpYH1rEF4Vl7YepGSF0La++6qDqmVWbz4fXnjfG2SjhHEsa8y+rAsuipx8zAm25Oh+NJNyI2Fz+B3A6zikfMKxdZUhnUd8ELV/0TUhHFYwQ2IRTwITcbAMtCqh6+zGtBLNMjRc/6An5EfFzilOVRpvnJUpc4s= 2022-01-20T20:32:52.668 INFO:teuthology.kill: smithi135.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAvOwkMsdXL8b9AVVBmPOkfHOy1umyJLoRKpu+g3cocYTaU9fLY9kTrCKc0M7lUk2gojXVyc8btugEMsgnqUxKujDRPtwEXPHS6zgCZlJyaxloq7a1ZGMM74V4fPJjHzyzEaMkpPE2ujPaNEgHCkTgoWjpGdBNtv/TgKFzC1ZtYPrxppbVqG77FNFPUkBZOeQjbMaIg5XfmPl4Ht96oS6+EtrxWznFZ5rFExu4/SXfIHv1xN9GpKaa8mUF7AM1ladxI2dpEc9LMIHgP+5oejTvAqefsWWO5mLUv3TLZjGzlu2hQYv1zyElzJL/nywGCfdppxqes0SUamIdjgvZXkPS4qH/o5W2d24DYpB9seyqjAkPnmCYAncxWtsOBeAnhRs+hvbDzjVCZheIEwTgcTE/4mPH2RwIdmkGYTkSg0QqF5zU2EH7ZQxoVLWfimmxC+k3R5PE9QESqeOycZp8yn55JVAgfeiUbLKyPvdTYdRnftRtw4Fb4h8+Ck6FNenRas8= 2022-01-20T20:32:52.698 INFO:teuthology.kill:2022-01-20 20:32:52,697.697 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-20T20:32:52.702 INFO:teuthology.kill:2022-01-20 20:32:52,702.702 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-20T20:32:52.782 INFO:teuthology.kill:2022-01-20 20:32:52,781.781 INFO:teuthology.orchestra.console:Power off smithi135 2022-01-20T20:32:52.795 INFO:teuthology.kill:2022-01-20 20:32:52,795.795 INFO:teuthology.orchestra.console:Power off smithi121 2022-01-20T20:33:01.158 INFO:teuthology.kill:2022-01-20 20:33:01,158.158 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-01-20T20:33:01.305 INFO:teuthology.kill:2022-01-20 20:33:01,304.304 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com 2022-01-20T20:33:05.265 INFO:teuthology.kill:2022-01-20 20:33:05,265.265 INFO:teuthology.orchestra.console:Power off for smithi135 completed 2022-01-20T20:33:05.413 INFO:teuthology.kill:2022-01-20 20:33:05,413.413 INFO:teuthology.lock.ops:unlocked: smithi135.front.sepia.ceph.com 2022-01-20T20:35:05.778 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-20T20:35:05.871 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi121 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-20_17:38:01-rados-wip-yuri5-testing-2022-01-20-0652-pacific-distro-default-smithi/6629499 2022-01-20T20:35:05.872 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi135 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-20_17:38:01-rados-wip-yuri5-testing-2022-01-20-0652-pacific-distro-default-smithi/6629497