2022-01-12T17:59:29.671 INFO:root:teuthology version: 1.1.0-6fc20113 2022-01-12T17:59:29.794 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi087.front.sepia.ceph.com' reimaging 2022-01-12T17:59:29.795 INFO:teuthology.lock.ops:Updating [ubuntu@smithi087.front.sepia.ceph.com]: reset os type and version on server 2022-01-12T17:59:29.796 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2022-01-12T17:59:29.815 INFO:teuthology.lock.ops:Node 'ubuntu@smithi087.front.sepia.ceph.com' reimaging is complete 2022-01-12T17:59:29.815 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi133.front.sepia.ceph.com' reimaging 2022-01-12T17:59:29.815 INFO:teuthology.lock.ops:Updating [ubuntu@smithi133.front.sepia.ceph.com]: reset os type and version on server 2022-01-12T17:59:29.816 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-01-12T17:59:29.835 INFO:teuthology.lock.ops:Node 'ubuntu@smithi133.front.sepia.ceph.com' reimaging is complete 2022-01-12T17:59:30.030 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of rhel 8.4 2022-01-12T17:59:30.048 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of rhel 8.4 2022-01-12T17:59:30.380 INFO:teuthology.orchestra.console:Power off smithi133 2022-01-12T17:59:30.395 INFO:teuthology.orchestra.console:Power off smithi087 2022-01-12T17:59:38.760 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2022-01-12T17:59:38.766 INFO:teuthology.orchestra.console:Power off for smithi087 completed 2022-01-12T17:59:38.862 INFO:teuthology.orchestra.console:Power on smithi133 2022-01-12T17:59:38.874 INFO:teuthology.orchestra.console:Power on smithi087 2022-01-12T17:59:43.129 INFO:teuthology.orchestra.console:Power on for smithi087 completed 2022-01-12T17:59:43.230 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish 2022-01-12T17:59:47.235 INFO:teuthology.orchestra.console:Power on for smithi133 completed 2022-01-12T17:59:47.455 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish 2022-01-12T18:04:32.822 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-12T18:04:33.073 INFO:teuthology.provision.fog.smithi087:Node is ready 2022-01-12T18:04:33.089 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com 2022-01-12T18:04:33.147 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2022-01-12T18:04:33.491 INFO:teuthology.provision.fog.smithi087:Deploy complete! 2022-01-12T18:04:36.896 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-12T18:04:37.121 INFO:teuthology.provision.fog.smithi133:Node is ready 2022-01-12T18:04:37.137 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com 2022-01-12T18:04:37.193 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133 2022-01-12T18:04:37.533 INFO:teuthology.provision.fog.smithi133:Deploy complete! 2022-01-12T18:04:37.690 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com 2022-01-12T18:04:37.691 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-12T18:04:37.710 INFO:teuthology.lock.ops:Checking smithi133.front.sepia.ceph.com 2022-01-12T18:04:37.711 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-12T18:04:37.731 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server 2022-01-12T18:04:37.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2022-01-12T18:04:37.968 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2022-01-12T18:04:38.026 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2022-01-12T18:04:38.027 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.4 (Ootpa)" 2022-01-12T18:04:38.027 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2022-01-12T18:04:38.027 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2022-01-12T18:04:38.027 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.4" 2022-01-12T18:04:38.028 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2022-01-12T18:04:38.028 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-12T18:04:38.028 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2022-01-12T18:04:38.028 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-12T18:04:38.029 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2022-01-12T18:04:38.029 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-12T18:04:38.029 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-12T18:04:38.029 INFO:teuthology.orchestra.run.smithi087.stdout: 2022-01-12T18:04:38.030 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-12T18:04:38.030 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-12T18:04:38.030 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-12T18:04:38.030 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-12T18:04:38.032 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2022-01-12T18:04:38.077 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: set os type and version on server 2022-01-12T18:04:38.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-01-12T18:04:38.302 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2022-01-12T18:04:38.358 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Red Hat Enterprise Linux" 2022-01-12T18:04:38.358 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8.4 (Ootpa)" 2022-01-12T18:04:38.359 INFO:teuthology.orchestra.run.smithi133.stdout:ID="rhel" 2022-01-12T18:04:38.359 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="fedora" 2022-01-12T18:04:38.359 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8.4" 2022-01-12T18:04:38.360 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2022-01-12T18:04:38.360 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-12T18:04:38.361 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2022-01-12T18:04:38.361 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-12T18:04:38.362 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.redhat.com/" 2022-01-12T18:04:38.362 INFO:teuthology.orchestra.run.smithi133.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-12T18:04:38.363 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-12T18:04:38.363 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-01-12T18:04:38.363 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-12T18:04:38.364 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-12T18:04:38.364 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-12T18:04:38.365 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-12T18:04:38.366 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-01-12T18:04:38.474 INFO:teuthology.dispatcher.supervisor:Running job 6610509 2022-01-12T18:04:38.474 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6fc2011361437a9dfe4e45b50de224392eed8abc/virtualenv/bin/teuthology -v --owner scheduled_melissali@teuthology --archive /home/teuthworker/archive/melissali-2022-01-12_17:43:53-rados:cephadm-wip-swagner-testing-2022-01-11-1258-distro-basic-smithi/6610509 --name melissali-2022-01-12_17:43:53-rados:cephadm-wip-swagner-testing-2022-01-11-1258-distro-basic-smithi --description rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/melissali-2022-01-12_17:43:53-rados:cephadm-wip-swagner-testing-2022-01-11-1258-distro-basic-smithi/6610509/orig.config.yaml 2022-01-12T18:04:38.478 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/melissali-2022-01-12_17:43:53-rados:cephadm-wip-swagner-testing-2022-01-11-1258-distro-basic-smithi/6610509 2022-01-12T18:04:38.479 INFO:teuthology.dispatcher.supervisor:Job PID: 15622 2022-01-12T18:04:38.479 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-13T06:05:32.507 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-13T06:05:32.727 INFO:teuthology.kill:Killing Pids: {15622} 2022-01-13T06:05:33.899 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-13T06:05:33.900 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-13T06:05:33.900 INFO:teuthology.misc:Compressing logs... 2022-01-13T06:05:33.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2022-01-13T06:05:34.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-01-13T06:05:34.134 INFO:teuthology.orchestra.run.smithi087.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-13T06:05:34.365 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-13T06:05:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-13T06:05:34.397 INFO:teuthology.orchestra.run.smithi087.stderr:tar: Error is not recoverable: exiting now 2022-01-13T06:05:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-13T06:05:34.431 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2022-01-13T06:05:34.432 INFO:teuthology.misc:Compressing logs... 2022-01-13T06:05:34.660 INFO:teuthology.misc:Compressing logs... 2022-01-13T06:05:37.431 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-13T06:05:39.078 INFO:teuthology.orchestra.run.smithi133.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-13T06:05:40.696 INFO:teuthology.orchestra.run.smithi133.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-mon.smithi133.log: file size changed while zipping 2022-01-13T06:05:40.857 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-mon.smithi087.log: file size changed while zipping 2022-01-13T06:05:41.453 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-mgr.smithi087.cxmpax.log: file size changed while zipping 2022-01-13T06:05:43.977 INFO:teuthology.orchestra.run.smithi133.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-13T06:05:44.145 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-13T06:05:47.153 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-13T06:05:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-13T06:05:50.843 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-13T06:05:51.806 INFO:teuthology.orchestra.run.smithi133.stderr:gzip: /var/log/ceph/990c6452-73d3-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-13T06:06:23.635 INFO:teuthology.kill:No teuthology processes running 2022-01-13T06:06:24.869 INFO:teuthology.kill:Nuking machines: ['smithi087', 'smithi133'] 2022-01-13T06:06:25.828 INFO:teuthology.kill:2022-01-13 06:06:25,827.827 INFO:teuthology.nuke:targets: 2022-01-13T06:06:25.829 INFO:teuthology.kill: smithi087.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeoGrn7yP6dpK3Bjkx69GNaahJAxTgejPd9eDGGTCUdsIhmZ9yQIQpkyfl8Aq2wYFGrQ8Bln3vPSbfeb/EHR0/aHDvGN+5tVlUctidl0CZiLs5MQBNO7qCCnZKo7Jruav05mqiAlQbvPY8SHJwYfgyQFcuuEr1gC/ZPmmHpNDheHKzXF+01JnyYLPTCD2jf/ugjiNUSkNV0QffydcE5mJjwDAqExTpWCDv1Xn9XYZF0S9nYY719IjFZfU6+sHcN+OoXezf8Q8yHxmMBimENeBRfBxzlkxXFH4Fs+2ysyncIj9T3raOr4GIdKIFwtV3YBpvqCynylGHHEZM1pZ2woZfKyBtB4a8yy/nnL/DhhDyaI2piv6hoNMKzg4XcEJL1uPOQt6Jq9xQJPrznz4Kdxs6ykInvSw8Ff9bAC8WipNmBXPOp+8IYfC1i1XF76Yrx1af2G7/v576keRF0o/WaguOon5G1WanZW4nvoZN9bgylBS1CItHekCDdlWyQxtjWgs= 2022-01-13T06:06:25.829 INFO:teuthology.kill: smithi133.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC63POaZr+pXBbMo6Y2rOr2hMrrImQCQ397lyhxSyfhfhUvvUFBQpcfRjCER0duIiJ+1mgGt1XMUmbRlX8+Q+M22UyPB+w4dG7l1ujnQxoBYGmgmsC/jHpeYnqMLC498aD4phtl2koTeJ/zJ2xO1iB44PUGp43lhnoqp+j6XnAP+RXWxXpBc5R8LW1+nS4WfuA/RqFh4/S5OyujcUVjv8REKa2hvz0EPbOyObWUILqvpsjUKUHuRrBbxBh//y0RtdYJsg+eMcHBYgGijAr2J6ducDy+YvaNEHUmYmchJrQmYOnCEWTSHx0x/DIbzjzIVYfmk4o/bjkoFQrCWKKecxFMLpCp6jly14UJkqiMAyRpqpeUKqM96MgYngh0Ddd3zfthuVGMNr+ITc23lsF8PdSsbhWmTdd6qTkLMfMbDMmy239HESoLfcGgDmsCHTW7tFcdQWSRjvoBvTLYEYcUnxmnRHBT79/Xvu0MMsFMxoFKEgky6z25xWLT44BpXOLu8dU= 2022-01-13T06:06:25.855 INFO:teuthology.kill:2022-01-13 06:06:25,855.855 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-13T06:06:25.859 INFO:teuthology.kill:2022-01-13 06:06:25,859.859 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-13T06:06:25.935 INFO:teuthology.kill:2022-01-13 06:06:25,935.935 INFO:teuthology.orchestra.console:Power off smithi087 2022-01-13T06:06:25.950 INFO:teuthology.kill:2022-01-13 06:06:25,950.950 INFO:teuthology.orchestra.console:Power off smithi133 2022-01-13T06:06:34.314 INFO:teuthology.kill:2022-01-13 06:06:34,314.314 INFO:teuthology.orchestra.console:Power off for smithi087 completed 2022-01-13T06:06:34.468 INFO:teuthology.kill:2022-01-13 06:06:34,467.467 INFO:teuthology.lock.ops:unlocked: smithi087.front.sepia.ceph.com 2022-01-13T06:06:38.433 INFO:teuthology.kill:2022-01-13 06:06:38,433.433 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2022-01-13T06:06:38.581 INFO:teuthology.kill:2022-01-13 06:06:38,581.581 INFO:teuthology.lock.ops:unlocked: smithi133.front.sepia.ceph.com 2022-01-13T06:08:38.909 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-13T06:08:39.003 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi087 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-12_21:05:52-fs-wip-yuri3-testing-2022-01-07-0927-pacific-distro-default-smithi/6611424 2022-01-13T06:08:39.004 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi133 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-12_21:05:52-fs-wip-yuri3-testing-2022-01-07-0927-pacific-distro-default-smithi/6611424