2023-07-11T08:31:05.550 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-11T08:31:05.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:31:05.552 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:31:15.658 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fed6d6d0790>, signals=[15]) 2023-07-11T08:31:15.659 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging 2023-07-11T08:31:15.660 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server 2023-07-11T08:31:15.660 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2023-07-11T08:31:15.683 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /host {"name": "smithi033"} 2023-07-11T08:31:20.959 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-11T08:31:26.010 DEBUG:teuthology.provision.fog.smithi033:Requesting image smithi_centos_8.stream (ID 3) 2023-07-11T08:31:26.010 DEBUG:teuthology.provision.fog.smithi033:FOG request: PUT /host/138 {"imageID": 3} 2023-07-11T08:31:31.065 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of centos 8.stream 2023-07-11T08:31:31.065 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:31:36.147 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /tasktype {"name": "deploy"} 2023-07-11T08:31:41.185 DEBUG:teuthology.provision.fog.smithi033:FOG request: POST /host/138/task {"taskTypeID": 1} 2023-07-11T08:31:46.270 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:31:55.872 INFO:teuthology.orchestra.console:Power off smithi033 2023-07-11T08:31:55.873 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T08:32:00.899 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T08:32:01.022 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2023-07-11T08:32:01.123 INFO:teuthology.orchestra.console:Power on smithi033 2023-07-11T08:32:01.123 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-11T08:32:01.144 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T08:32:05.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T08:32:10.279 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T08:32:10.488 INFO:teuthology.orchestra.console:Power on for smithi033 completed 2023-07-11T08:32:10.590 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2023-07-11T08:32:10.590 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:32:25.724 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:32:42.360 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:33:02.456 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:33:22.553 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:33:42.643 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:34:02.732 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:34:22.832 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:34:42.708 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-07-11T08:34:47.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T08:34:55.856 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-07-11T08:35:01.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T08:35:04.912 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-07-11T08:35:10.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T08:35:14.004 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-07-11T08:35:20.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T08:36:20.068 WARNING:teuthology.provision.fog.smithi033:timed out 2023-07-11T08:36:26.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T08:36:26.320 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-11T08:36:26.322 DEBUG:teuthology.orchestra.run.smithi033:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-11T08:36:26.587 INFO:teuthology.provision.fog.smithi033:Node is ready 2023-07-11T08:36:26.587 DEBUG:teuthology.orchestra.run.smithi033:> hostname 2023-07-11T08:36:26.608 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com 2023-07-11T08:36:26.608 DEBUG:teuthology.orchestra.run.smithi033:> grep smithi033.front.sepia.ceph.com /etc/hosts 2023-07-11T08:36:26.669 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033 2023-07-11T08:36:26.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2023-07-11T08:36:26.778 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hosts 2023-07-11T08:36:26.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/172.21.15.33/172.21.15.33/g' /etc/hosts 2023-07-11T08:36:26.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hostname 2023-07-11T08:36:26.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2023-07-11T08:36:27.015 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2023-07-11T08:36:27.074 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2023-07-11T08:36:27.074 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8" 2023-07-11T08:36:27.074 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2023-07-11T08:36:27.074 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2023-07-11T08:36:27.074 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8" 2023-07-11T08:36:27.075 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8" 2023-07-11T08:36:27.075 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-11T08:36:27.075 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2023-07-11T08:36:27.075 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-11T08:36:27.075 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2023-07-11T08:36:27.075 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-11T08:36:27.075 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-11T08:36:27.075 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-11T08:36:27.076 INFO:teuthology.provision.fog.smithi033:Deploy complete! 2023-07-11T08:36:27.078 DEBUG:teuthology.parallel:result is None 2023-07-11T08:36:27.283 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com 2023-07-11T08:36:27.284 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-11T08:36:27.308 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server 2023-07-11T08:36:27.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-07-11T08:36:27.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T08:36:27.376 DEBUG:teuthology.orchestra.run.smithi033:> true 2023-07-11T08:36:27.444 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-07-11T08:36:27.444 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2023-07-11T08:36:27.501 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2023-07-11T08:36:27.502 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2023-07-11T08:36:27.559 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2023-07-11T08:36:27.560 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8" 2023-07-11T08:36:27.560 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2023-07-11T08:36:27.561 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2023-07-11T08:36:27.561 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8" 2023-07-11T08:36:27.561 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8" 2023-07-11T08:36:27.561 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-11T08:36:27.561 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2023-07-11T08:36:27.561 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-11T08:36:27.561 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2023-07-11T08:36:27.562 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-11T08:36:27.562 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-11T08:36:27.562 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-11T08:36:27.562 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2023-07-11T08:36:32.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:36:32.489 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:36:37.586 INFO:teuthology.dispatcher.supervisor:Running job 7333528 2023-07-11T08:36:37.587 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2023-07-11_07:33:56-orch:cephadm-guits-cv-refactor-osd-objectstore-distro-default-smithi/7333528 --name gabrioux-2023-07-11_07:33:56-orch:cephadm-guits-cv-refactor-osd-objectstore-distro-default-smithi --description orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs} -- /home/teuthworker/archive/gabrioux-2023-07-11_07:33:56-orch:cephadm-guits-cv-refactor-osd-objectstore-distro-default-smithi/7333528/orig.config.yaml 2023-07-11T08:36:37.590 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2023-07-11_07:33:56-orch:cephadm-guits-cv-refactor-osd-objectstore-distro-default-smithi/7333528 2023-07-11T08:36:37.591 INFO:teuthology.dispatcher.supervisor:Job PID: 9125 2023-07-11T08:36:37.591 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-11T08:38:37.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:38:37.690 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:40:42.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:40:42.870 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:42:48.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:42:48.050 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:44:52.181 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:44:52.182 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:46:57.365 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:46:57.366 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:49:02.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:49:02.564 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:51:07.698 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:51:07.698 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:53:07.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:53:07.886 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:55:13.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:55:13.061 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:57:16.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T08:57:16.422 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T08:57:17.160 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-11T08:57:17.214 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-11T08:57:17.215 INFO:teuthology.nuke:Checking targets against current locks 2023-07-11T08:57:17.258 DEBUG:teuthology.nuke:shortname: smithi033 2023-07-11T08:57:17.261 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-11T08:57:17.287 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-07-11_07:33:56-orch:cephadm-guits-cv-refactor-osd-objectstore-distro-default-smithi/7333528', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-11 08:30:54.767471', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOIzxfzlGsb2Yokh47r6Av/jVUsdGifaoAlXiBmGphR+gvtdIxKnpBMpE+ow56uDW1cFi4iH534voNMUib/I97eb30ljoDbbK6RVnyjNpVFc3fgM584fSJwwaA/SP/DXpOBb6zpa/wu0IJsZMSFpV5nIZHYYDY2MELwgT1d0CjjRTfxvlO/DZVrVXSYtIYPm5NgWC0QKj6PihMTuncLFupi3MlFazH/tYxyk3rGPw6IiaVV+7I0fQhZC+ODSrGzEc5Ugy3aBmWOIm7KXKxoj7pks260QAGJz55XqQdaKIDw7UQU8eWuW4rkZPHpgHrUhW6a2/lFK7NMtUKW52WF0+nOHkmUCnyfNdBlogwKETr1i+8O4yQCmYAL3x+biirJkf2DHtWfqRdqmGgCfE/S+i7kiMvKagqLFtbVvQL1qKBkQdP9xfdXe8+L+t+FMY7BeYMH0WCD7OKmA23IbzD9M5eXEoaHccI5SAFKvpcM6fu2BOgtMJG2eU/AT7ed8T0SMc='} 2023-07-11T08:57:17.341 INFO:teuthology.orchestra.console:Power off smithi033 2023-07-11T08:57:17.341 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T08:57:17.370 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T08:57:17.495 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2023-07-11T08:57:17.652 INFO:teuthology.lock.ops:unlocked: smithi033.front.sepia.ceph.com