2023-11-10T06:38:26.493 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-10T06:38:26.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:38:26.595 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fda048ec670>, signals=[15]) 2023-11-10T06:38:26.596 INFO:teuthology.lock.ops:Start node 'smithi037.front.sepia.ceph.com' reimaging 2023-11-10T06:38:26.596 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: reset os type and version on server 2023-11-10T06:38:26.597 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2023-11-10T06:38:26.616 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /host {"name": "smithi037"} 2023-11-10T06:38:26.687 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-10T06:38:26.734 DEBUG:teuthology.provision.fog.smithi037:Requesting image smithi_centos_9.stream (ID 10) 2023-11-10T06:38:26.735 DEBUG:teuthology.provision.fog.smithi037:FOG request: PUT /host/142 {"imageID": 10} 2023-11-10T06:38:26.782 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of centos 9.stream 2023-11-10T06:38:26.782 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:38:26.838 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /tasktype {"name": "deploy"} 2023-11-10T06:38:26.877 DEBUG:teuthology.provision.fog.smithi037:FOG request: POST /host/142/task {"taskTypeID": 1} 2023-11-10T06:38:26.937 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:38:27.017 INFO:teuthology.orchestra.console.smithi037:Power off 2023-11-10T06:38:27.018 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T06:38:27.042 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2023-11-10T06:38:27.043 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T06:38:27.070 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2023-11-10T06:38:31.074 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T06:38:31.184 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2023-11-10T06:38:35.186 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T06:38:35.294 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-11-10T06:38:35.395 INFO:teuthology.orchestra.console.smithi037:Power off completed 2023-11-10T06:38:35.496 INFO:teuthology.orchestra.console.smithi037:Power on 2023-11-10T06:38:35.496 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-10T06:38:35.520 DEBUG:teuthology.orchestra.console.smithi037:power on output: Chassis Power Control: Up/On 2023-11-10T06:38:35.521 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T06:38:35.543 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-11-10T06:38:39.546 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T06:38:39.654 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-11-10T06:38:43.659 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T06:38:43.768 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2023-11-10T06:38:43.869 INFO:teuthology.orchestra.console.smithi037:Power on completed 2023-11-10T06:38:43.969 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish 2023-11-10T06:38:43.970 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:38:59.055 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:39:14.125 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:39:29.218 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:39:44.314 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:39:59.434 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:40:14.546 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:40:29.666 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:40:44.764 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:40:59.874 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-11-10T06:40:59.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T06:41:18.462 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2023-11-10T06:41:24.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T06:41:27.546 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2023-11-10T06:41:33.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T06:41:36.602 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2023-11-10T06:41:42.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T06:41:45.662 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2023-11-10T06:41:51.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T06:42:51.727 WARNING:teuthology.provision.fog.smithi037:timed out 2023-11-10T06:42:57.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T06:42:58.000 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-10T06:42:58.001 DEBUG:teuthology.orchestra.run.smithi037:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-10T06:42:58.355 INFO:teuthology.provision.fog.smithi037:Node is ready 2023-11-10T06:42:58.356 DEBUG:teuthology.orchestra.run.smithi037:> hostname 2023-11-10T06:42:58.371 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com 2023-11-10T06:42:58.371 DEBUG:teuthology.orchestra.run.smithi037:> grep smithi037.front.sepia.ceph.com /etc/hosts 2023-11-10T06:42:58.426 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037 2023-11-10T06:42:58.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname smithi037 2023-11-10T06:42:58.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/smithi037.front.sepia.ceph.com/smithi037/g' /etc/hosts 2023-11-10T06:42:58.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/172.21.15.37/172.21.15.37/g' /etc/hosts 2023-11-10T06:42:58.639 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/smithi037.front.sepia.ceph.com/smithi037/g' /etc/hostname 2023-11-10T06:42:58.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname smithi037 2023-11-10T06:42:58.781 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2023-11-10T06:42:58.836 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2023-11-10T06:42:58.836 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2023-11-10T06:42:58.836 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2023-11-10T06:42:58.836 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2023-11-10T06:42:58.837 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2023-11-10T06:42:58.837 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2023-11-10T06:42:58.837 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-10T06:42:58.837 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2023-11-10T06:42:58.837 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2023-11-10T06:42:58.838 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-10T06:42:58.838 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2023-11-10T06:42:58.838 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T06:42:58.838 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-10T06:42:58.838 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T06:42:58.839 INFO:teuthology.provision.fog.smithi037:Deploy complete! 2023-11-10T06:42:58.841 DEBUG:teuthology.parallel:result is None 2023-11-10T06:42:58.966 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com 2023-11-10T06:42:58.967 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-10T06:42:59.002 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server 2023-11-10T06:42:59.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-11-10T06:42:59.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T06:42:59.099 DEBUG:teuthology.orchestra.run.smithi037:> true 2023-11-10T06:42:59.164 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-11-10T06:42:59.165 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2023-11-10T06:42:59.219 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2023-11-10T06:42:59.219 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2023-11-10T06:42:59.274 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2023-11-10T06:42:59.275 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2023-11-10T06:42:59.275 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2023-11-10T06:42:59.275 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2023-11-10T06:42:59.275 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2023-11-10T06:42:59.276 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2023-11-10T06:42:59.276 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-10T06:42:59.276 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2023-11-10T06:42:59.276 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2023-11-10T06:42:59.277 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-10T06:42:59.277 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2023-11-10T06:42:59.277 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T06:42:59.277 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-10T06:42:59.277 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T06:42:59.278 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2023-11-10T06:42:59.301 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:42:59.386 INFO:teuthology.dispatcher.supervisor:Running job 7453781 2023-11-10T06:42:59.387 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_nmordech@teuthology --archive /home/teuthworker/archive/nmordech-2023-11-10_06:35:04-rados:rest-main-distro-default-smithi/7453781 --name nmordech-2023-11-10_06:35:04-rados:rest-main-distro-default-smithi --description rados:rest/{mgr-restful supported-random-distro$/{centos_latest overrides}} -- /home/teuthworker/archive/nmordech-2023-11-10_06:35:04-rados:rest-main-distro-default-smithi/7453781/orig.config.yaml 2023-11-10T06:42:59.390 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nmordech-2023-11-10_06:35:04-rados:rest-main-distro-default-smithi/7453781 2023-11-10T06:42:59.391 INFO:teuthology.dispatcher.supervisor:Job PID: 342974 2023-11-10T06:42:59.391 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-10T06:44:59.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:46:59.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:48:59.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:51:00.010 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:53:00.187 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:55:00.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:57:00.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T06:59:00.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T07:01:00.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T07:03:01.045 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T07:05:01.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T07:07:01.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T07:09:01.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T07:11:01.749 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T07:11:01.829 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-10T07:11:01.892 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-10T07:11:01.892 INFO:teuthology.nuke:Checking targets against current locks 2023-11-10T07:11:01.936 DEBUG:teuthology.nuke:shortname: smithi037 2023-11-10T07:11:01.937 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-10T07:11:01.958 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/nmordech-2023-11-10_06:35:04-rados:rest-main-distro-default-smithi/7453781', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-10 06:38:25.725456', 'locked_by': 'scheduled_nmordech@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKuWxYC55QGJo6xcy7UwOl2TUugmpSjK7jbC+ZEW6BOTQKoCVemXrezsgIaPpO2kcE+CBUAByNKSlpTxrIPgiXs='} 2023-11-10T07:11:02.015 INFO:teuthology.orchestra.console.smithi037:Power off 2023-11-10T07:11:02.015 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T07:11:02.037 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2023-11-10T07:11:02.038 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T07:11:02.059 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-11-10T07:11:02.160 INFO:teuthology.orchestra.console.smithi037:Power off completed 2023-11-10T07:11:02.311 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com