2022-01-13T09:37:19.385 INFO:root:teuthology version: 1.1.0-6fc20113
2022-01-13T09:37:19.512 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging
2022-01-13T09:37:19.514 INFO:teuthology.lock.ops:Updating [ubuntu@smithi052.front.sepia.ceph.com]: reset os type and version on server
2022-01-13T09:37:19.515 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2022-01-13T09:37:19.535 INFO:teuthology.lock.ops:Node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging is complete
2022-01-13T09:37:19.536 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging
2022-01-13T09:37:19.536 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server
2022-01-13T09:37:19.536 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2022-01-13T09:37:19.554 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete
2022-01-13T09:37:19.752 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of rhel 8.4
2022-01-13T09:37:19.849 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of rhel 8.4
2022-01-13T09:37:20.258 INFO:teuthology.orchestra.console:Power off smithi052
2022-01-13T09:37:20.369 INFO:teuthology.orchestra.console:Power off smithi116
2022-01-13T09:37:28.632 INFO:teuthology.orchestra.console:Power off for smithi052 completed
2022-01-13T09:37:28.733 INFO:teuthology.orchestra.console:Power on smithi052
2022-01-13T09:37:32.844 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2022-01-13T09:37:32.946 INFO:teuthology.orchestra.console:Power on smithi116
2022-01-13T09:37:37.099 INFO:teuthology.orchestra.console:Power on for smithi052 completed
2022-01-13T09:37:37.201 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish
2022-01-13T09:37:41.308 INFO:teuthology.orchestra.console:Power on for smithi116 completed
2022-01-13T09:37:41.409 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish
2022-01-13T09:41:53.218 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-13T09:42:08.614 INFO:teuthology.provision.fog.smithi116:Node is ready
2022-01-13T09:42:08.633 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com
2022-01-13T09:42:08.690 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116
2022-01-13T09:42:09.025 INFO:teuthology.provision.fog.smithi116:Deploy complete!
2022-01-13T09:42:11.762 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-13T09:42:11.974 INFO:teuthology.provision.fog.smithi052:Node is ready
2022-01-13T09:42:11.989 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052.front.sepia.ceph.com
2022-01-13T09:42:12.046 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52 smithi052.front.sepia.ceph.com smithi052
2022-01-13T09:42:12.357 INFO:teuthology.provision.fog.smithi052:Deploy complete!
2022-01-13T09:42:12.517 INFO:teuthology.lock.ops:Checking smithi052.front.sepia.ceph.com
2022-01-13T09:42:12.518 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-13T09:42:12.536 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com
2022-01-13T09:42:12.537 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-13T09:42:12.554 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: set os type and version on server
2022-01-13T09:42:12.555 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2022-01-13T09:42:12.779 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64
2022-01-13T09:42:12.833 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="Red Hat Enterprise Linux"
2022-01-13T09:42:12.834 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="8.4 (Ootpa)"
2022-01-13T09:42:12.834 INFO:teuthology.orchestra.run.smithi052.stdout:ID="rhel"
2022-01-13T09:42:12.834 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="fedora"
2022-01-13T09:42:12.834 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="8.4"
2022-01-13T09:42:12.835 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el8"
2022-01-13T09:42:12.835 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-01-13T09:42:12.835 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31"
2022-01-13T09:42:12.835 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-01-13T09:42:12.835 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://www.redhat.com/"
2022-01-13T09:42:12.836 INFO:teuthology.orchestra.run.smithi052.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-01-13T09:42:12.836 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-13T09:42:12.836 INFO:teuthology.orchestra.run.smithi052.stdout:
2022-01-13T09:42:12.836 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-13T09:42:12.837 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-01-13T09:42:12.837 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-01-13T09:42:12.837 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-01-13T09:42:12.838 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2022-01-13T09:42:12.858 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server
2022-01-13T09:42:12.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com'
2022-01-13T09:42:13.092 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64
2022-01-13T09:42:13.147 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Red Hat Enterprise Linux"
2022-01-13T09:42:13.147 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8.4 (Ootpa)"
2022-01-13T09:42:13.147 INFO:teuthology.orchestra.run.smithi116.stdout:ID="rhel"
2022-01-13T09:42:13.148 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="fedora"
2022-01-13T09:42:13.148 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8.4"
2022-01-13T09:42:13.148 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8"
2022-01-13T09:42:13.148 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-01-13T09:42:13.148 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31"
2022-01-13T09:42:13.149 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-01-13T09:42:13.149 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.redhat.com/"
2022-01-13T09:42:13.149 INFO:teuthology.orchestra.run.smithi116.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-01-13T09:42:13.149 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-13T09:42:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:
2022-01-13T09:42:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-13T09:42:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-01-13T09:42:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-01-13T09:42:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-01-13T09:42:13.151 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2022-01-13T09:42:13.268 INFO:teuthology.dispatcher.supervisor:Running job 6611026
2022-01-13T09:42:13.268 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6fc2011361437a9dfe4e45b50de224392eed8abc/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-01-12_19:45:04-rados-WIP_GBH_NCB_expand_device_fix_1-distro-basic-smithi/6611026 --name benhanokh-2022-01-12_19:45:04-rados-WIP_GBH_NCB_expand_device_fix_1-distro-basic-smithi --description rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} -- /home/teuthworker/archive/benhanokh-2022-01-12_19:45:04-rados-WIP_GBH_NCB_expand_device_fix_1-distro-basic-smithi/6611026/orig.config.yaml
2022-01-13T09:42:13.272 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-01-12_19:45:04-rados-WIP_GBH_NCB_expand_device_fix_1-distro-basic-smithi/6611026
2022-01-13T09:42:13.273 INFO:teuthology.dispatcher.supervisor:Job PID: 8812
2022-01-13T09:42:13.273 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-13T21:42:33.677 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-01-13T21:42:33.847 INFO:teuthology.kill:Killing Pids: {8812}
2022-01-13T21:42:35.014 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['host.a', 'client.0']
2022-01-13T21:42:35.015 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['host.b', 'client.1']
2022-01-13T21:42:35.016 INFO:teuthology.misc:Compressing logs...
2022-01-13T21:42:35.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2022-01-13T21:42:35.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com'
2022-01-13T21:42:35.244 INFO:teuthology.orchestra.run.smithi052.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-13T21:42:35.425 INFO:teuthology.orchestra.run.smithi116.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-13T21:42:35.467 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-13T21:42:35.468 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now
2022-01-13T21:42:35.507 INFO:teuthology.orchestra.run.smithi116.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-13T21:42:35.507 INFO:teuthology.orchestra.run.smithi116.stderr:tar: Error is not recoverable: exiting now
2022-01-13T21:42:35.509 INFO:teuthology.misc:Compressing logs...
2022-01-13T21:42:35.770 INFO:teuthology.misc:Compressing logs...
2022-01-13T21:43:07.085 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-01-13T21:43:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-01-13T21:43:13.098 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-mon.smithi116.log: file size changed while zipping
2022-01-13T21:43:29.479 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-mon.smithi052.log: file size changed while zipping
2022-01-13T21:43:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-01-13T21:43:33.586 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-mgr.smithi052.vsjghm.log: file size changed while zipping
2022-01-13T21:44:02.967 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-01-13T21:44:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-01-13T21:44:33.872 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-01-13T21:44:39.831 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-01-13T21:44:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-client.rgw.foo.smithi052.ttrfbr.log: file size changed while zipping
2022-01-13T21:45:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/a2fe916e-7456-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-01-13T21:46:24.802 INFO:teuthology.kill:No teuthology processes running
2022-01-13T21:46:25.989 INFO:teuthology.kill:Nuking machines: ['smithi052', 'smithi116']
2022-01-13T21:46:27.014 INFO:teuthology.kill:2022-01-13 21:46:27,014.014 INFO:teuthology.nuke:targets:
2022-01-13T21:46:27.015 INFO:teuthology.kill:  smithi052.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCRFtvy/hiI9ySFPFP5G7qDlbz3EmPyA+Uh5Fdsbod9wT9qVef42aNYAFMLccUboV065PIB0g5BZxy1qpKYgV1yLINM+SQJYxma3HYOuLpBWYtKAPH0qYRvYdlpgR6CX/9kPUqysHKCBjwkcgfDodCHGHeXCwKiVnKomfo6m3g+n559bS3uF3evYP4MSDYxUG1PqgkY2TJ5TAUDbA0zYsxb3946qY2vzPd/5Te4i2GOTb1LQVlDSZp4TaLRoFyVqS79nehj243kWwtid4DEfwmCZZA19xqFkzfrkPd215dxsVXI3VHfQ+pIzqkrYXoTUPVqLqD3hDGf3YzNf5Qvp5xiBuJ3ST7L0tjJmIWg9lJKbKI327Ifh9nKbLG2XsEdAnazHwFgIdoasjvTISSyjwO6MsjOikS/pX/0+Ql4hjWMyaTDPvqgCPduEF0ZLoTrzkszPyh0hrxnSfuTPLEQV/jSa7eE3GRR60/7GP1TgMAJih4miLEu1SBeGgWNM67UFuk=
2022-01-13T21:46:27.015 INFO:teuthology.kill:  smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR9Zz9e9wzcUH9POv5pz4z3lhynbSv0fwVy2UYboAYu4qokP9GF4je1PjOfJmyZhMvPpj0XVnDnaN5Bfh4VFwvdfxfGX1ySmcZaMrnMr+oDWFecOsBivdy6s7t+mDC7na+dK4zFy9zygQoqCPLRaoM6kHjEF2qYIYnBuZuURlhqGgGIl2onEECrApkSQl5aHNnjx8Qqbd4qo1fF6zKiLs27WaSJbYRm2moBV2WA1HKekd5tpmNU3cjO4CeQSd5bBTA+ouAov4GzQjPZrcufXgBiaYknLIAoA0W1DIP+JRWh6EcHAMKmF3DBu9kLmOxyBQ1HUTrWFv2gQHlIBEn1mXjh7T8U/Vn+5P6tB+IA17o1O1UZZaVPRkaG8E/D+RVQvUGKe8XnbKI0dHZFr3tRB5iZDphabOVoZbIga+58uv8eyAoAOr1ckYpGiLV3rkwe3Nn8gzZgAoAVAfpGBC3EXGzvyvXQaG07pzafouzgffeF8WY/7KoAuS2zwkN15tOkTE=
2022-01-13T21:46:27.040 INFO:teuthology.kill:2022-01-13 21:46:27,039.039 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-13T21:46:27.043 INFO:teuthology.kill:2022-01-13 21:46:27,043.043 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-13T21:46:27.116 INFO:teuthology.kill:2022-01-13 21:46:27,116.116 INFO:teuthology.orchestra.console:Power off smithi116
2022-01-13T21:46:27.130 INFO:teuthology.kill:2022-01-13 21:46:27,129.129 INFO:teuthology.orchestra.console:Power off smithi052
2022-01-13T21:46:35.481 INFO:teuthology.kill:2022-01-13 21:46:35,480.480 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2022-01-13T21:46:35.629 INFO:teuthology.kill:2022-01-13 21:46:35,628.628 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com
2022-01-13T21:46:39.601 INFO:teuthology.kill:2022-01-13 21:46:39,600.600 INFO:teuthology.orchestra.console:Power off for smithi052 completed
2022-01-13T21:46:39.748 INFO:teuthology.kill:2022-01-13 21:46:39,748.748 INFO:teuthology.lock.ops:unlocked: smithi052.front.sepia.ceph.com
2022-01-13T21:48:40.062 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-13T21:48:40.163 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi052 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-13_18:06:52-rados-wip-yuri3-testing-2022-01-13-0809-distro-default-smithi/6614532
2022-01-13T21:48:40.163 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi116 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-13_18:06:52-rados-wip-yuri3-testing-2022-01-13-0809-distro-default-smithi/6614528