2022-01-20T13:22:35.893 INFO:root:teuthology version: 1.1.0-3ff86a5d 2022-01-20T13:22:36.092 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi104.front.sepia.ceph.com' reimaging 2022-01-20T13:22:36.093 INFO:teuthology.lock.ops:Updating [ubuntu@smithi104.front.sepia.ceph.com]: reset os type and version on server 2022-01-20T13:22:36.094 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2022-01-20T13:22:36.118 INFO:teuthology.lock.ops:Node 'ubuntu@smithi104.front.sepia.ceph.com' reimaging is complete 2022-01-20T13:22:36.119 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi159.front.sepia.ceph.com' reimaging 2022-01-20T13:22:36.120 INFO:teuthology.lock.ops:Updating [ubuntu@smithi159.front.sepia.ceph.com]: reset os type and version on server 2022-01-20T13:22:36.121 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-01-20T13:22:36.146 INFO:teuthology.lock.ops:Node 'ubuntu@smithi159.front.sepia.ceph.com' reimaging is complete 2022-01-20T13:22:36.350 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of centos 8.3 2022-01-20T13:22:36.386 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 8.3 2022-01-20T13:22:36.768 INFO:teuthology.orchestra.console:Power off smithi104 2022-01-20T13:22:36.885 INFO:teuthology.orchestra.console:Power off smithi159 2022-01-20T13:22:45.251 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-01-20T13:22:45.352 INFO:teuthology.orchestra.console:Power on smithi159 2022-01-20T13:22:49.266 INFO:teuthology.orchestra.console:Power off for smithi104 completed 2022-01-20T13:22:49.367 INFO:teuthology.orchestra.console:Power on smithi104 2022-01-20T13:22:49.609 INFO:teuthology.orchestra.console:Power on for smithi159 completed 2022-01-20T13:22:49.710 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2022-01-20T13:22:57.739 INFO:teuthology.orchestra.console:Power on for smithi104 completed 2022-01-20T13:22:57.840 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish 2022-01-20T13:26:57.189 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-20T13:27:17.409 INFO:teuthology.provision.fog.smithi104:Node is ready 2022-01-20T13:27:17.428 INFO:teuthology.orchestra.run.smithi104.stdout:smithi104.front.sepia.ceph.com 2022-01-20T13:27:17.485 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.104 smithi104.front.sepia.ceph.com smithi104 2022-01-20T13:27:17.834 INFO:teuthology.provision.fog.smithi104:Deploy complete! 2022-01-20T13:27:36.702 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-20T13:27:36.996 INFO:teuthology.provision.fog.smithi159:Node is ready 2022-01-20T13:27:37.011 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2022-01-20T13:27:37.067 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2022-01-20T13:27:37.396 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2022-01-20T13:27:37.545 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com 2022-01-20T13:27:37.546 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-20T13:27:37.565 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2022-01-20T13:27:37.565 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-20T13:27:37.584 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server 2022-01-20T13:27:37.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2022-01-20T13:27:37.853 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2022-01-20T13:27:37.909 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Linux" 2022-01-20T13:27:37.910 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="8" 2022-01-20T13:27:37.910 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2022-01-20T13:27:37.910 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2022-01-20T13:27:37.911 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="8" 2022-01-20T13:27:37.911 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el8" 2022-01-20T13:27:37.911 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-20T13:27:37.911 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2022-01-20T13:27:37.912 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-20T13:27:37.912 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2022-01-20T13:27:37.912 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-20T13:27:37.912 INFO:teuthology.orchestra.run.smithi104.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-20T13:27:37.912 INFO:teuthology.orchestra.run.smithi104.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-20T13:27:37.914 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2022-01-20T13:27:37.933 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2022-01-20T13:27:37.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-01-20T13:27:38.186 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2022-01-20T13:27:38.241 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Linux" 2022-01-20T13:27:38.241 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8" 2022-01-20T13:27:38.241 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2022-01-20T13:27:38.241 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2022-01-20T13:27:38.242 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8" 2022-01-20T13:27:38.242 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2022-01-20T13:27:38.242 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-20T13:27:38.242 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2022-01-20T13:27:38.243 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-20T13:27:38.243 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2022-01-20T13:27:38.243 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-20T13:27:38.243 INFO:teuthology.orchestra.run.smithi159.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-20T13:27:38.244 INFO:teuthology.orchestra.run.smithi159.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-20T13:27:38.245 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-01-20T13:27:38.342 INFO:teuthology.dispatcher.supervisor:Running job 6624029 2022-01-20T13:27:38.342 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3ff86a5dbaafecdb6414fa2fea970bb44e836829/virtualenv/bin/teuthology -v --owner scheduled_ifed01@teuthology --archive /home/teuthworker/archive/ifed01-2022-01-18_11:04:04-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6624029 --name ifed01-2022-01-18_11:04:04-rados-wip-ifed-faster-rm-p2-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} -- /home/teuthworker/archive/ifed01-2022-01-18_11:04:04-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6624029/orig.config.yaml 2022-01-20T13:27:38.346 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ifed01-2022-01-18_11:04:04-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6624029 2022-01-20T13:27:38.346 INFO:teuthology.dispatcher.supervisor:Job PID: 29549 2022-01-20T13:27:38.347 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-20T19:58:07.983 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-20T19:58:08.164 INFO:teuthology.kill:Killing Pids: {29549} 2022-01-20T19:58:09.318 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-20T19:58:09.319 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-20T19:58:09.319 INFO:teuthology.misc:Compressing logs... 2022-01-20T19:58:09.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2022-01-20T19:58:09.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-01-20T19:58:09.577 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-20T19:58:09.783 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-20T19:58:09.814 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-20T19:58:09.815 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2022-01-20T19:58:09.843 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-20T19:58:09.843 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2022-01-20T19:58:09.845 INFO:teuthology.misc:Compressing logs... 2022-01-20T19:58:10.043 INFO:teuthology.misc:Compressing logs... 2022-01-20T19:58:13.533 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-mon.smithi159.log: file size changed while zipping 2022-01-20T19:58:17.979 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-20T19:58:20.764 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-20T19:58:25.137 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-20T19:58:29.671 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-20T19:58:34.674 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-20T19:58:36.457 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-mgr.smithi104.bhdzaj.log: file size changed while zipping 2022-01-20T19:58:36.517 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-20T19:58:45.229 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-20T19:58:45.725 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-mon.smithi104.log: file size changed while zipping 2022-01-20T19:58:52.895 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/ff602c70-79f5-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-20T19:59:09.656 INFO:teuthology.kill:No teuthology processes running 2022-01-20T19:59:10.795 INFO:teuthology.kill:Nuking machines: ['smithi104', 'smithi159'] 2022-01-20T19:59:11.813 INFO:teuthology.kill:2022-01-20 19:59:11,813.813 INFO:teuthology.nuke:targets: 2022-01-20T19:59:11.814 INFO:teuthology.kill: smithi104.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRLKkCtjIjOkz0B4UCKIGncFT6X2tR7cCp9usPo4QU5lSG+7EVd8KEw1zN2Tkgb/V+F7U1bekf8gc6f2i4ypRa2JQeXsQRRL8owtbFkWjSf7ZmMHUDd1QvKinNcuWJkm5r9lEH9WpH86E+1FT05yb3+DAbqQh8u/diNojbQdi+awl05VRhpDGRZIdnz8ZpFO+S1fpmoeOwwSRwz4HJi0WyivnyI2KuyRvF+PQSdDBTZOyXuHImRLccZW3y1YWAn/1Vk2GxuTMUFCo+265PmlSS/yN/9UhOOw5CbdDuX09miDPehhhP++z++IuSmBAUFuNImC4CcbyGY4vXM+2GLYduQf/Ze1k/6dcECtZVlVqTazG7UytQ9k+rfGE0wXVoK/09QKBBiYzCqDg4ieHfDc2RY06CBQOL11S74xALvrxwArF2y/cAwZocif8eO580bLiUSIa67VnujlirI5loEZS1f/9ffHflteL8+oJ2pQS0Nfb+NUPPSyFtskJKC4xzi4U= 2022-01-20T19:59:11.815 INFO:teuthology.kill: smithi159.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpX/COk6kVT/o7m+/wP30eQRYrbhDX75pDR68M4mMGxxPvJZVQGtHzRWkDfEtgfbwS7QmID+1EFPLX/d3GAofSaNvuzc9X4v0Bme37jWZFvQKpKZcCGxGR8z6W2FjNgVlJOdP2Hkg/fCg0pa2q+As8pRw1scIiEczxGaJbBgMy91CPGRztIid1wlgktJo2GQ85WryevZSCFR6ChuwDHetBghjYOagEVMshJxT3L63MEu+HZR0lF/cxKO+RPLzUrrRNO4JnGQj/0yGiUEUkm7DN6eHNZtSGU56ZFEx3ziKugEmGHS+LYXjNSjZazOuaLhxjf1rmGuEZgD/sSjKRJlWL8E6xJ2A15pw+ZtZ6YL3sRq1oFwZlsyxL8zJIIpVJWNS8dAhXvxsfCnoQVgh1LsgzhturQRiKubQ2A0RKZjKXt5Nj5dUfrhFwY2zdnmOIGEGxoJK0z/veNrzRBNUTTDMwDZgpY8I9lA0R+AriNwPUroxEG9ImH8Xb+P84jutkkGE= 2022-01-20T19:59:11.841 INFO:teuthology.kill:2022-01-20 19:59:11,841.841 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-20T19:59:11.847 INFO:teuthology.kill:2022-01-20 19:59:11,846.846 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-20T19:59:11.925 INFO:teuthology.kill:2022-01-20 19:59:11,925.925 INFO:teuthology.orchestra.console:Power off smithi159 2022-01-20T19:59:11.938 INFO:teuthology.kill:2022-01-20 19:59:11,938.938 INFO:teuthology.orchestra.console:Power off smithi104 2022-01-20T19:59:20.295 INFO:teuthology.kill:2022-01-20 19:59:20,294.294 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-01-20T19:59:20.440 INFO:teuthology.kill:2022-01-20 19:59:20,439.439 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com 2022-01-20T19:59:24.415 INFO:teuthology.kill:2022-01-20 19:59:24,414.414 INFO:teuthology.orchestra.console:Power off for smithi104 completed 2022-01-20T19:59:24.561 INFO:teuthology.kill:2022-01-20 19:59:24,561.561 INFO:teuthology.lock.ops:unlocked: smithi104.front.sepia.ceph.com 2022-01-20T20:01:24.878 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-20T20:01:24.986 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi104 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-20_17:34:11-fs-wip-yuri5-testing-2022-01-20-0652-pacific-distro-default-smithi/6629435 2022-01-20T20:01:24.987 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi159 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-20_17:34:11-fs-wip-yuri5-testing-2022-01-20-0652-pacific-distro-default-smithi/6629435