2023-06-29T17:30:30.401 INFO:root:teuthology version: 0.0.1.dev134+ge25bf39 2023-06-29T17:30:30.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:30:30.403 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:30:30.522 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9a1fa745e0>, signals=[15]) 2023-06-29T17:30:30.523 INFO:teuthology.lock.ops:Start node 'smithi006.front.sepia.ceph.com' reimaging 2023-06-29T17:30:30.524 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: reset os type and version on server 2023-06-29T17:30:30.524 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-06-29T17:30:30.547 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /host {"name": "smithi006"} 2023-06-29T17:30:30.625 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-06-29T17:30:30.660 DEBUG:teuthology.provision.fog.smithi006:Requesting image smithi_centos_8.stream (ID 3) 2023-06-29T17:30:30.660 DEBUG:teuthology.provision.fog.smithi006:FOG request: PUT /host/112 {"imageID": 3} 2023-06-29T17:30:30.699 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of centos 8.stream 2023-06-29T17:30:30.699 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:30:30.768 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /tasktype {"name": "deploy"} 2023-06-29T17:30:30.806 DEBUG:teuthology.provision.fog.smithi006:FOG request: POST /host/112/task {"taskTypeID": 1} 2023-06-29T17:30:30.862 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:30:30.964 INFO:teuthology.orchestra.console:Power off smithi006 2023-06-29T17:30:30.965 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-29T17:30:30.987 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T17:30:35.017 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T17:30:39.128 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T17:30:43.240 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T17:30:43.450 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2023-06-29T17:30:43.551 INFO:teuthology.orchestra.console:Power on smithi006 2023-06-29T17:30:43.551 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-29T17:30:43.574 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T17:30:47.599 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T17:30:47.808 INFO:teuthology.orchestra.console:Power on for smithi006 completed 2023-06-29T17:30:47.909 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish 2023-06-29T17:30:47.909 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:31:02.989 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:31:18.072 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:31:33.140 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:31:48.204 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:32:03.276 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:32:18.339 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:32:33.410 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:32:48.476 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:33:03.544 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:33:18.606 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:33:33.672 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-06-29T17:33:33.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-29T17:33:52.112 WARNING:teuthology.provision.fog.smithi006:[Errno None] Unable to connect to port 22 on 172.21.15.6 2023-06-29T17:33:58.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-29T17:34:01.200 WARNING:teuthology.provision.fog.smithi006:[Errno None] Unable to connect to port 22 on 172.21.15.6 2023-06-29T17:34:07.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-29T17:34:10.256 WARNING:teuthology.provision.fog.smithi006:[Errno None] Unable to connect to port 22 on 172.21.15.6 2023-06-29T17:34:16.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-29T17:35:16.320 WARNING:teuthology.provision.fog.smithi006:timed out 2023-06-29T17:35:22.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-29T17:35:22.536 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-29T17:35:22.537 DEBUG:teuthology.orchestra.run.smithi006:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-29T17:35:22.783 INFO:teuthology.provision.fog.smithi006:Node is ready 2023-06-29T17:35:22.783 DEBUG:teuthology.orchestra.run.smithi006:> hostname 2023-06-29T17:35:22.802 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com 2023-06-29T17:35:22.803 DEBUG:teuthology.orchestra.run.smithi006:> grep smithi006.front.sepia.ceph.com /etc/hosts 2023-06-29T17:35:22.860 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006 2023-06-29T17:35:22.861 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname smithi006 2023-06-29T17:35:22.969 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/smithi006.front.sepia.ceph.com/smithi006/g' /etc/hosts 2023-06-29T17:35:22.999 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/172.21.15.6/172.21.15.6/g' /etc/hosts 2023-06-29T17:35:23.066 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/smithi006.front.sepia.ceph.com/smithi006/g' /etc/hostname 2023-06-29T17:35:23.136 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname smithi006 2023-06-29T17:35:23.203 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-06-29T17:35:23.261 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2023-06-29T17:35:23.261 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8" 2023-06-29T17:35:23.261 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2023-06-29T17:35:23.261 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2023-06-29T17:35:23.262 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8" 2023-06-29T17:35:23.262 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-06-29T17:35:23.262 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-29T17:35:23.262 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-06-29T17:35:23.262 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-29T17:35:23.262 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2023-06-29T17:35:23.262 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-29T17:35:23.263 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-29T17:35:23.263 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-29T17:35:23.263 INFO:teuthology.provision.fog.smithi006:Deploy complete! 2023-06-29T17:35:23.265 DEBUG:teuthology.parallel:result is None 2023-06-29T17:35:23.415 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com 2023-06-29T17:35:23.416 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-29T17:35:23.438 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server 2023-06-29T17:35:23.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-06-29T17:35:23.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-29T17:35:23.500 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-06-29T17:35:23.575 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2023-06-29T17:35:23.632 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2023-06-29T17:35:23.632 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-06-29T17:35:23.688 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2023-06-29T17:35:23.688 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8" 2023-06-29T17:35:23.688 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2023-06-29T17:35:23.688 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2023-06-29T17:35:23.688 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8" 2023-06-29T17:35:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-06-29T17:35:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-29T17:35:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-06-29T17:35:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-29T17:35:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2023-06-29T17:35:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-29T17:35:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-29T17:35:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-29T17:35:23.690 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-06-29T17:35:23.714 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:35:23.714 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:35:23.929 INFO:teuthology.dispatcher.supervisor:Running job 7321734 2023-06-29T17:35:23.930 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-29_15:47:19-rados-wip-yuri6-testing-2023-06-28-0739-reef-distro-default-smithi/7321734 --name yuriw-2023-06-29_15:47:19-rados-wip-yuri6-testing-2023-06-28-0739-reef-distro-default-smithi --description rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} -- /home/teuthworker/archive/yuriw-2023-06-29_15:47:19-rados-wip-yuri6-testing-2023-06-28-0739-reef-distro-default-smithi/7321734/orig.config.yaml 2023-06-29T17:35:23.933 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-06-29_15:47:19-rados-wip-yuri6-testing-2023-06-28-0739-reef-distro-default-smithi/7321734 2023-06-29T17:35:23.934 INFO:teuthology.dispatcher.supervisor:Job PID: 2743 2023-06-29T17:35:23.934 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-06-29T17:37:24.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:37:24.034 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:39:24.217 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:39:24.217 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:41:24.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:41:24.390 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:43:24.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:43:24.562 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:45:24.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:45:24.731 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:47:24.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:47:24.906 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:49:25.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:49:25.048 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:51:25.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:51:25.214 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:53:25.382 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:53:25.383 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:55:25.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:55:25.560 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:57:25.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:57:25.730 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T17:59:25.906 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T17:59:25.907 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T18:01:26.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T18:01:26.070 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T18:01:26.150 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-06-29T18:01:26.213 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-06-29T18:01:26.213 INFO:teuthology.nuke:Checking targets against current locks 2023-06-29T18:01:26.259 DEBUG:teuthology.nuke:shortname: smithi006 2023-06-29T18:01:26.260 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-29T18:01:26.284 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-29_15:47:19-rados-wip-yuri6-testing-2023-06-28-0739-reef-distro-default-smithi/7321734', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-29 17:30:29.642977', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1osZz7Opi2MzuXZz2jkszK9KJiTIzhSsRmRQT0+67ulFgW9eySvC9MeVe8EWGA3OL/9Rxf/noh50tWzC9l1m4yGCwgN9HzmKTVmYeR/tp3vvL1tFl6NmAzZfOQdXokR+hC5UfUNAcoV+c8O8MpyzWT191Or2ePxr4h4NLOZ3/irgEBy79UWagQzRiVDHKcsUv8LKDKalr643HupkCTeiFWoLxSXhqkKHpRwgoj+DOgKmGV/rUN1AtrG5qzewPN9uQqWv33X7xLkCP81JZ1jCYTDH82yJrDHbDlZFfZt8YkC7j2lcrhm4ZslVtg9AU0fPh3TaNN5sOsEoczVHdZDGfWbecpMeN4kFtHLkUSNyAXEsGrO7so/SOQSV0u+r2Lx/Q/lge76DU9k1jNfyn5+lIdCD7gfbzpK4tR5uRHx8qTvbG8Vs3C81wMkg89z6YESi8JbJRwj/r2EwewDkxDBFvKoQRby9WL0weLHFv9XSq8I/SbOR1TAcITq6wkR5pXG0='} 2023-06-29T18:01:26.337 INFO:teuthology.orchestra.console:Power off smithi006 2023-06-29T18:01:26.338 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-29T18:01:26.358 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T18:01:26.479 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2023-06-29T18:01:26.629 INFO:teuthology.lock.ops:unlocked: smithi006.front.sepia.ceph.com