2022-01-25T23:07:40.191 INFO:root:teuthology version: 1.1.0-75f1bcd2 2022-01-25T23:07:40.312 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging 2022-01-25T23:07:40.314 INFO:teuthology.lock.ops:Updating [ubuntu@smithi149.front.sepia.ceph.com]: reset os type and version on server 2022-01-25T23:07:40.314 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2022-01-25T23:07:40.332 INFO:teuthology.lock.ops:Node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging is complete 2022-01-25T23:07:40.333 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi195.front.sepia.ceph.com' reimaging 2022-01-25T23:07:40.333 INFO:teuthology.lock.ops:Updating [ubuntu@smithi195.front.sepia.ceph.com]: reset os type and version on server 2022-01-25T23:07:40.334 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2022-01-25T23:07:40.354 INFO:teuthology.lock.ops:Node 'ubuntu@smithi195.front.sepia.ceph.com' reimaging is complete 2022-01-25T23:07:40.559 INFO:teuthology.provision.fog.smithi195:Scheduling deploy of centos 8.3 2022-01-25T23:07:40.563 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of centos 8.3 2022-01-25T23:07:40.964 INFO:teuthology.orchestra.console:Power off smithi149 2022-01-25T23:07:40.979 INFO:teuthology.orchestra.console:Power off smithi195 2022-01-25T23:07:49.343 INFO:teuthology.orchestra.console:Power off for smithi195 completed 2022-01-25T23:07:49.444 INFO:teuthology.orchestra.console:Power on smithi195 2022-01-25T23:07:53.447 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2022-01-25T23:07:53.548 INFO:teuthology.orchestra.console:Power on smithi149 2022-01-25T23:07:57.816 INFO:teuthology.orchestra.console:Power on for smithi195 completed 2022-01-25T23:07:57.918 INFO:teuthology.provision.fog.smithi195:Waiting for deploy to finish 2022-01-25T23:08:01.923 INFO:teuthology.orchestra.console:Power on for smithi149 completed 2022-01-25T23:08:02.024 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish 2022-01-25T23:12:29.844 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-25T23:12:30.133 INFO:teuthology.provision.fog.smithi195:Node is ready 2022-01-25T23:12:30.147 INFO:teuthology.orchestra.run.smithi195.stdout:smithi195.front.sepia.ceph.com 2022-01-25T23:12:30.203 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.15.195 smithi195.front.sepia.ceph.com smithi195 2022-01-25T23:12:30.528 INFO:teuthology.provision.fog.smithi195:Deploy complete! 2022-01-25T23:12:42.974 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-25T23:12:43.271 INFO:teuthology.provision.fog.smithi149:Node is ready 2022-01-25T23:12:43.328 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com 2022-01-25T23:12:43.384 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149 2022-01-25T23:12:43.719 INFO:teuthology.provision.fog.smithi149:Deploy complete! 2022-01-25T23:12:43.867 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com 2022-01-25T23:12:43.868 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-25T23:12:43.896 INFO:teuthology.lock.ops:Checking smithi195.front.sepia.ceph.com 2022-01-25T23:12:43.897 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-25T23:12:43.916 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server 2022-01-25T23:12:43.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2022-01-25T23:12:44.178 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2022-01-25T23:12:44.232 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Linux" 2022-01-25T23:12:44.232 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8" 2022-01-25T23:12:44.232 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2022-01-25T23:12:44.233 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2022-01-25T23:12:44.233 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8" 2022-01-25T23:12:44.233 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8" 2022-01-25T23:12:44.233 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-25T23:12:44.233 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2022-01-25T23:12:44.234 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-25T23:12:44.234 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2022-01-25T23:12:44.234 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-25T23:12:44.234 INFO:teuthology.orchestra.run.smithi149.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-25T23:12:44.235 INFO:teuthology.orchestra.run.smithi149.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-25T23:12:44.236 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2022-01-25T23:12:44.252 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: set os type and version on server 2022-01-25T23:12:44.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2022-01-25T23:12:44.523 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2022-01-25T23:12:44.577 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Linux" 2022-01-25T23:12:44.577 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="8" 2022-01-25T23:12:44.577 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2022-01-25T23:12:44.577 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2022-01-25T23:12:44.578 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="8" 2022-01-25T23:12:44.578 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el8" 2022-01-25T23:12:44.578 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-25T23:12:44.578 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2022-01-25T23:12:44.578 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-25T23:12:44.579 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2022-01-25T23:12:44.579 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-25T23:12:44.579 INFO:teuthology.orchestra.run.smithi195.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-25T23:12:44.579 INFO:teuthology.orchestra.run.smithi195.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-25T23:12:44.580 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2022-01-25T23:12:44.666 INFO:teuthology.dispatcher.supervisor:Running job 6637702 2022-01-25T23:12:44.666 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_75f1bcd237a8b6022305cd53ba87e106d45aa665/virtualenv/bin/teuthology -v --owner scheduled_ifed01@teuthology --archive /home/teuthworker/archive/ifed01-2022-01-24_11:40:23-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6637702 --name ifed01-2022-01-24_11:40:23-rados-wip-ifed-faster-rm-p2-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} -- /home/teuthworker/archive/ifed01-2022-01-24_11:40:23-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6637702/orig.config.yaml 2022-01-25T23:12:44.669 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ifed01-2022-01-24_11:40:23-rados-wip-ifed-faster-rm-p2-distro-default-smithi/6637702 2022-01-25T23:12:44.670 INFO:teuthology.dispatcher.supervisor:Job PID: 7066 2022-01-25T23:12:44.670 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-26T05:43:14.035 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-26T05:43:14.243 INFO:teuthology.kill:Killing Pids: {7066} 2022-01-26T05:43:15.419 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-26T05:43:15.420 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-26T05:43:15.421 INFO:teuthology.misc:Compressing logs... 2022-01-26T05:43:15.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2022-01-26T05:43:15.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2022-01-26T05:43:15.676 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-26T05:43:15.948 INFO:teuthology.orchestra.run.smithi195.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-26T05:43:15.978 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-26T05:43:15.979 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2022-01-26T05:43:16.016 INFO:teuthology.orchestra.run.smithi195.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-26T05:43:16.017 INFO:teuthology.orchestra.run.smithi195.stderr:tar: Error is not recoverable: exiting now 2022-01-26T05:43:16.018 INFO:teuthology.misc:Compressing logs... 2022-01-26T05:43:16.222 INFO:teuthology.misc:Compressing logs... 2022-01-26T05:43:23.503 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-26T05:43:24.132 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-26T05:43:31.337 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-26T05:43:32.432 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-26T05:43:39.297 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-26T05:43:40.824 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-26T05:43:48.080 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-26T05:43:49.988 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-mon.smithi149.log: file size changed while zipping 2022-01-26T05:43:51.525 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-mon.smithi195.log: file size changed while zipping 2022-01-26T05:43:57.184 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-26T05:43:58.882 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/97327902-7e35-11ec-8c35-001a4aab830c/ceph-mgr.smithi149.ecbjfz.log: file size changed while zipping 2022-01-26T05:44:15.601 INFO:teuthology.kill:No teuthology processes running 2022-01-26T05:44:17.173 INFO:teuthology.kill:Nuking machines: ['smithi149', 'smithi195'] 2022-01-26T05:44:18.243 INFO:teuthology.kill:2022-01-26 05:44:18,243.243 INFO:teuthology.nuke:targets: 2022-01-26T05:44:18.244 INFO:teuthology.kill: smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxIrMngxKmG08Ru2RdRP4Div0XSY+zpQxHfn54mlBmX3EZLsHS74eIUUb50eT6VZySFkZ1Cnhjkg9Zo7s4Obss35O2WfdSrAI4irYU7esgN8V/MLk3HYmlkduIIV1qP89ENGJ32ViDHiw5I1Bc5+eXkZCBawdt+hRuNR1UsQ5N2gNyyx+HUQE8ILlsxb/SI4DRaQ/Qt2gLs68QMOLsHfzOMmOXeJVA73XtYXsMLRBMQ/Vey94lDX2JpOjVICprcT0+JUs1dhbDLUAuur51mG7h/dpWepybH5xTGH27JI8e7t2IO3RPcDbEJ7jsr821NfxqLtNMiPAVyqjM0FsbwO1MGs9QejN17m1REe7CmwSZL7CRpye9Yw4KbNyihMbtu6nAIMwQ1WHgO6QiX0UVswIs2d+qOxU4QuZ/aXE9J7Hc1Jpc1Ufmi/3co8/g5D10AbSazPtgyf2Cdik7XFVrhwPU3QEpi+1toU4euVAxd6rBl/XOjZsNSQiTGnTq/dSuQs8= 2022-01-26T05:44:18.244 INFO:teuthology.kill: smithi195.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC592stTkAXzueJq5qijurwR4kbyHSO9M+4Dm1OgbABRGmFUju5jRfXfiJGblqvfvnatQDHEG0vTLitrViXS8GCoNAzAu6hEwqSN4roOEHKG1/ms0UiOk2a6j/9bDCrTWB4eCTsYVWUZxju0q1sGv5FauJhBY7PfxlD1hNb9LCfqASkSkCwWipg9e8S9QJx67etpgoYk/MTQ4AmpZFbbXHB1tunRcoBI8l1KOfZ5/Y6GkntOS+46Qq/a28NhINp0NGs5yGXCIz7y1dQOgn4C4Pq1WvqWkvRnydmsyIfdw1+0HA2jmW8ZV3PJBJXT4qOdkj1eDk92e/mBsZB4O7ppyS2TB5h3Gut3VwqwIoiyhyWtcxKkVaU8RmP3i0xaadYH8ChU3pxcYJNNfT7tzUDH5yBYAkB5xP0udq9RzSBx9frBX9HV8qo6op5HguSBo6ptGp+kXf+yS5ZKwQuZuj5f3ZloQYdaQNWkA4NcRnGdehS8LqlBcrZYbEjQ+nYQc72zMk= 2022-01-26T05:44:18.273 INFO:teuthology.kill:2022-01-26 05:44:18,273.273 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T05:44:18.285 INFO:teuthology.kill:2022-01-26 05:44:18,285.285 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T05:44:18.365 INFO:teuthology.kill:2022-01-26 05:44:18,365.365 INFO:teuthology.orchestra.console:Power off smithi149 2022-01-26T05:44:18.387 INFO:teuthology.kill:2022-01-26 05:44:18,386.386 INFO:teuthology.orchestra.console:Power off smithi195 2022-01-26T05:44:26.767 INFO:teuthology.kill:2022-01-26 05:44:26,767.767 INFO:teuthology.orchestra.console:Power off for smithi195 completed 2022-01-26T05:44:26.934 INFO:teuthology.kill:2022-01-26 05:44:26,933.933 INFO:teuthology.lock.ops:unlocked: smithi195.front.sepia.ceph.com 2022-01-26T05:44:30.877 INFO:teuthology.kill:2022-01-26 05:44:30,877.877 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2022-01-26T05:44:31.026 INFO:teuthology.kill:2022-01-26 05:44:31,026.026 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com 2022-01-26T05:46:31.408 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-26T05:46:31.511 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi149 but it was locked by another job: /home/teuthworker/archive/adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi/6641095 2022-01-26T05:46:31.511 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi195 but it was locked by another job: /home/teuthworker/archive/adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi/6641097