2022-01-26T06:09:45.370 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-26T06:09:45.487 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi142.front.sepia.ceph.com' reimaging 2022-01-26T06:09:45.488 INFO:teuthology.lock.ops:Updating [ubuntu@smithi142.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T06:09:45.488 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2022-01-26T06:09:45.506 INFO:teuthology.lock.ops:Node 'ubuntu@smithi142.front.sepia.ceph.com' reimaging is complete 2022-01-26T06:09:45.507 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging 2022-01-26T06:09:45.507 INFO:teuthology.lock.ops:Updating [ubuntu@smithi155.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T06:09:45.507 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2022-01-26T06:09:45.525 INFO:teuthology.lock.ops:Node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging is complete 2022-01-26T06:09:45.704 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of centos 8.3 2022-01-26T06:09:45.723 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of centos 8.3 2022-01-26T06:09:46.018 INFO:teuthology.orchestra.console:Power off smithi142 2022-01-26T06:09:46.056 INFO:teuthology.orchestra.console:Power off smithi155 2022-01-26T06:09:58.507 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2022-01-26T06:09:58.521 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2022-01-26T06:09:58.609 INFO:teuthology.orchestra.console:Power on smithi142 2022-01-26T06:09:58.623 INFO:teuthology.orchestra.console:Power on smithi155 2022-01-26T06:10:02.865 INFO:teuthology.orchestra.console:Power on for smithi142 completed 2022-01-26T06:10:02.872 INFO:teuthology.orchestra.console:Power on for smithi155 completed 2022-01-26T06:10:02.966 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish 2022-01-26T06:10:02.973 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2022-01-26T06:14:34.950 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T06:14:35.232 INFO:teuthology.provision.fog.smithi142:Node is ready 2022-01-26T06:14:35.287 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142.front.sepia.ceph.com 2022-01-26T06:14:35.342 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142 smithi142.front.sepia.ceph.com smithi142 2022-01-26T06:14:35.663 INFO:teuthology.provision.fog.smithi142:Deploy complete! 2022-01-26T06:14:49.983 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T06:14:50.312 INFO:teuthology.provision.fog.smithi155:Node is ready 2022-01-26T06:14:50.327 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2022-01-26T06:14:50.383 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2022-01-26T06:14:50.708 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2022-01-26T06:14:50.861 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com 2022-01-26T06:14:50.862 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T06:14:50.880 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2022-01-26T06:14:50.881 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T06:14:50.898 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server 2022-01-26T06:14:50.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2022-01-26T06:14:51.169 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2022-01-26T06:14:51.224 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Linux" 2022-01-26T06:14:51.224 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8" 2022-01-26T06:14:51.224 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2022-01-26T06:14:51.225 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2022-01-26T06:14:51.225 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8" 2022-01-26T06:14:51.225 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2022-01-26T06:14:51.225 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-26T06:14:51.226 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2022-01-26T06:14:51.226 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-26T06:14:51.226 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2022-01-26T06:14:51.226 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-26T06:14:51.227 INFO:teuthology.orchestra.run.smithi142.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-26T06:14:51.227 INFO:teuthology.orchestra.run.smithi142.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-26T06:14:51.228 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2022-01-26T06:14:51.245 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2022-01-26T06:14:51.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2022-01-26T06:14:51.511 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2022-01-26T06:14:51.565 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Linux" 2022-01-26T06:14:51.565 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8" 2022-01-26T06:14:51.565 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2022-01-26T06:14:51.566 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2022-01-26T06:14:51.566 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8" 2022-01-26T06:14:51.566 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2022-01-26T06:14:51.566 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Linux 8" 2022-01-26T06:14:51.567 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2022-01-26T06:14:51.567 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-26T06:14:51.567 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2022-01-26T06:14:51.567 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-26T06:14:51.568 INFO:teuthology.orchestra.run.smithi155.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-26T06:14:51.568 INFO:teuthology.orchestra.run.smithi155.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-26T06:14:51.568 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2022-01-26T06:14:51.656 INFO:teuthology.dispatcher.supervisor:Running job 6641138 2022-01-26T06:14:51.657 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi/6641138 --name adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi --description rados:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} -- /home/teuthworker/archive/adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi/6641138/orig.config.yaml 2022-01-26T06:14:51.660 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi/6641138 2022-01-26T06:14:51.661 INFO:teuthology.dispatcher.supervisor:Job PID: 632 2022-01-26T06:14:51.661 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-26T12:45:21.434 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-26T12:45:21.600 INFO:teuthology.kill:Killing Pids: {632} 2022-01-26T12:45:22.784 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-26T12:45:22.785 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-26T12:45:22.785 INFO:teuthology.misc:Compressing logs... 2022-01-26T12:45:22.786 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2022-01-26T12:45:23.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2022-01-26T12:45:23.081 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-26T12:45:23.371 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-26T12:45:23.403 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-26T12:45:23.404 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2022-01-26T12:45:23.430 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-26T12:45:23.430 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2022-01-26T12:45:23.431 INFO:teuthology.misc:Compressing logs... 2022-01-26T12:45:23.668 INFO:teuthology.misc:Compressing logs... 2022-01-26T12:45:36.026 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-26T12:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-26T12:45:49.619 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-26T12:45:52.543 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-26T12:46:02.678 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-26T12:46:06.638 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-26T12:46:10.853 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-mgr.smithi142.cqxsel.log: file size changed while zipping 2022-01-26T12:46:17.619 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-26T12:46:19.787 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-mon.smithi142.log: file size changed while zipping 2022-01-26T12:46:20.884 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-mon.smithi155.log: file size changed while zipping 2022-01-26T12:46:34.016 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/9587c6b2-7e70-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-26T12:47:00.690 INFO:teuthology.kill:No teuthology processes running 2022-01-26T12:47:01.850 INFO:teuthology.kill:Nuking machines: ['smithi142', 'smithi155'] 2022-01-26T12:47:02.828 INFO:teuthology.kill:2022-01-26 12:47:02,828.828 INFO:teuthology.nuke:targets: 2022-01-26T12:47:02.829 INFO:teuthology.kill: smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXbHK6/YCnKEbOaaWdmKWuh77gbJmUpnf2Fi5QbfYMec8HBIP0XN6mnacvCQ0bjVpK8ypkLLFfvcZ1PdrL/ew8PV01MNk2oHpYZrubmhhU5Cue4GQviDC24JscIGWbYSr9xo7IhORwHspcgThAhYF0bEyBASbDntx54GY0uMOz/X0c5be5ESF0zuw7zpv04+kRh3550ePnPfcLlMHa68t3VC2b/72LUkegxUZFw2u9+eEACx5eB7iwElLQSW/dD9BKqYZt+QqzyWCYcOgmZ7W2MY2UhrRnggca1gI4M47XVYVu0/V5S4bRhwG1aajP+W6ARxiCPTdt91A3KhqHtifZl+IMf82Zhyc0xOEbLnfg+3/n/edp3Cy71hEgWZp36qd6Nc2QMmpHMbJ93k2LEGnsfRYgm5ACY2RJ0fDN+pQrV3utRNCHn5RfrbuRw/7nv3H8aNjD/ONh581x/mqL7GNazLRl4QRyUxBMY/Jf1y/rIyM0WKGwpSid9wkRk3jLS6s= 2022-01-26T12:47:02.830 INFO:teuthology.kill: smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzzTEdyePLkDh0XLgZP7ldZaKkL0Z6wYZnUW/YKk61uJRd+/rRGpb6uppsy7pHFnbX/RjMFydosseflO6YGaDhQOQvrdJbIAXqdWW+t+oRaHhtsSlvSH8ClPnzeYNNypo2UXMzUk//BRsJWi0/Aoq/dAThX82SioUvgdTTo12TcPEW9oWFpb0ZN8oe85FNVJJ4gXpHdamINO5ITs2z6ZKSA6U8ZHq3f44Y7tMXpx6J5MO8/ZqP1q56w12DzM6DEOofa7RmykPYhB2Sw2QQgwQSCwjl0CxtbBpVvR0VBBPier75m4E3hyjNWjveGqzKc7cUM/uMFJyMgn0NfN1q27GFs4BvzXrloR58A6Ky592HMTfcJCBbnvkE+TqRKuiFS25Vrus1+z1g4WPKeFO8vh8cjMy/6werMCrlC5ugv4zxQF4Eeq0WBiVh5B1SjkL3Geddym6zZK92KqhRLVh31AuW5XWO5oeC30emlouspUkI6OKvaDy/yHV7YUEjKXulc0E= 2022-01-26T12:47:02.855 INFO:teuthology.kill:2022-01-26 12:47:02,855.855 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T12:47:02.858 INFO:teuthology.kill:2022-01-26 12:47:02,858.858 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T12:47:02.931 INFO:teuthology.kill:2022-01-26 12:47:02,931.931 INFO:teuthology.orchestra.console:Power off smithi155 2022-01-26T12:47:02.945 INFO:teuthology.kill:2022-01-26 12:47:02,945.945 INFO:teuthology.orchestra.console:Power off smithi142 2022-01-26T12:47:15.413 INFO:teuthology.kill:2022-01-26 12:47:15,413.413 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2022-01-26T12:47:15.420 INFO:teuthology.kill:2022-01-26 12:47:15,420.420 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2022-01-26T12:47:15.566 INFO:teuthology.kill:2022-01-26 12:47:15,565.565 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com 2022-01-26T12:47:15.570 INFO:teuthology.kill:2022-01-26 12:47:15,570.570 INFO:teuthology.lock.ops:unlocked: smithi142.front.sepia.ceph.com 2022-01-26T12:49:15.878 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-26T12:49:15.972 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi142 but it was locked by another job: /home/teuthworker/archive/mchangir-2022-01-26_12:36:36-fs-wip-mchangir-mgr-snap_schedule-retesting-branch-distro-default-smithi/6641382 2022-01-26T12:49:15.973 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi155 but it was locked by another job: /home/teuthworker/archive/mchangir-2022-01-26_12:36:36-fs-wip-mchangir-mgr-snap_schedule-retesting-branch-distro-default-smithi/6641382