2023-08-08T22:09:25.241 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-08T22:09:25.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:09:25.386 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc208f02280>, signals=[15]) 2023-08-08T22:09:25.388 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging 2023-08-08T22:09:25.388 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server 2023-08-08T22:09:25.389 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-08-08T22:09:25.410 INFO:teuthology.lock.ops:Start node 'smithi079.front.sepia.ceph.com' reimaging 2023-08-08T22:09:25.410 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: reset os type and version on server 2023-08-08T22:09:25.410 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2023-08-08T22:09:25.412 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /host {"name": "smithi050"} 2023-08-08T22:09:25.435 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /host {"name": "smithi079"} 2023-08-08T22:09:25.482 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-08-08T22:09:25.502 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-08-08T22:09:25.518 DEBUG:teuthology.provision.fog.smithi050:Requesting image smithi_centos_9.stream (ID 10) 2023-08-08T22:09:25.519 DEBUG:teuthology.provision.fog.smithi050:FOG request: PUT /host/155 {"imageID": 10} 2023-08-08T22:09:25.542 DEBUG:teuthology.provision.fog.smithi079:Requesting image smithi_centos_9.stream (ID 10) 2023-08-08T22:09:25.543 DEBUG:teuthology.provision.fog.smithi079:FOG request: PUT /host/184 {"imageID": 10} 2023-08-08T22:09:25.564 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 9.stream 2023-08-08T22:09:25.565 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:09:25.589 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of centos 9.stream 2023-08-08T22:09:25.589 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:09:25.632 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /tasktype {"name": "deploy"} 2023-08-08T22:09:25.646 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /tasktype {"name": "deploy"} 2023-08-08T22:09:25.671 DEBUG:teuthology.provision.fog.smithi050:FOG request: POST /host/155/task {"taskTypeID": 1} 2023-08-08T22:09:25.681 DEBUG:teuthology.provision.fog.smithi079:FOG request: POST /host/184/task {"taskTypeID": 1} 2023-08-08T22:09:25.788 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:09:25.805 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:09:25.928 INFO:teuthology.orchestra.console.smithi079:Power off 2023-08-08T22:09:25.929 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T22:09:25.945 INFO:teuthology.orchestra.console.smithi050:Power off 2023-08-08T22:09:25.945 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T22:09:25.954 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2023-08-08T22:09:25.955 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T22:09:25.967 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2023-08-08T22:09:25.967 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T22:09:25.978 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-08-08T22:09:25.989 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2023-08-08T22:09:26.080 INFO:teuthology.orchestra.console.smithi079:Power off completed 2023-08-08T22:09:26.091 INFO:teuthology.orchestra.console.smithi050:Power off completed 2023-08-08T22:09:26.181 INFO:teuthology.orchestra.console.smithi079:Power on 2023-08-08T22:09:26.182 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-08T22:09:26.192 INFO:teuthology.orchestra.console.smithi050:Power on 2023-08-08T22:09:26.193 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-08T22:09:26.204 DEBUG:teuthology.orchestra.console.smithi079:power on output: Chassis Power Control: Up/On 2023-08-08T22:09:26.205 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T22:09:26.218 DEBUG:teuthology.orchestra.console.smithi050:power on output: Chassis Power Control: Up/On 2023-08-08T22:09:26.219 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T22:09:26.229 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-08-08T22:09:26.241 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2023-08-08T22:09:30.232 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T22:09:30.241 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T22:09:30.342 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-08-08T22:09:30.350 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2023-08-08T22:09:30.451 INFO:teuthology.orchestra.console.smithi050:Power on completed 2023-08-08T22:09:30.552 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2023-08-08T22:09:30.552 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:09:34.344 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T22:09:34.457 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-08-08T22:09:34.558 INFO:teuthology.orchestra.console.smithi079:Power on completed 2023-08-08T22:09:34.659 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish 2023-08-08T22:09:34.659 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:09:45.652 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:09:49.748 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:10:00.729 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:10:04.826 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:10:15.816 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:10:19.904 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:10:30.903 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:10:34.984 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:10:45.976 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:10:50.061 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:11:01.064 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:11:05.133 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:11:16.140 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:11:20.204 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:11:31.224 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:11:35.280 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:11:46.300 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:11:50.352 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:12:01.368 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-08-08T22:12:01.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:04.497 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-08-08T22:12:05.420 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-08-08T22:12:05.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:08.564 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-08T22:12:10.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:13.556 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-08-08T22:12:14.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:17.648 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-08T22:12:19.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:22.640 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-08-08T22:12:23.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:26.704 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-08T22:12:28.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:31.696 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-08-08T22:12:32.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:35.760 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-08T22:12:37.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:40.756 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-08-08T22:12:41.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:44.820 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-08T22:12:46.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:49.844 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-08-08T22:12:50.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:53.876 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-08-08T22:12:55.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:12:59.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:13:55.909 WARNING:teuthology.provision.fog.smithi050:timed out 2023-08-08T22:13:59.884 WARNING:teuthology.provision.fog.smithi079:timed out 2023-08-08T22:14:01.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:14:02.113 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-08T22:14:02.114 DEBUG:teuthology.orchestra.run.smithi050:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-08T22:14:02.414 INFO:teuthology.provision.fog.smithi050:Node is ready 2023-08-08T22:14:02.415 DEBUG:teuthology.orchestra.run.smithi050:> hostname 2023-08-08T22:14:02.429 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2023-08-08T22:14:02.430 DEBUG:teuthology.orchestra.run.smithi050:> grep smithi050.front.sepia.ceph.com /etc/hosts 2023-08-08T22:14:02.487 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2023-08-08T22:14:02.488 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2023-08-08T22:14:02.591 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hosts 2023-08-08T22:14:02.622 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/172.21.15.50/172.21.15.50/g' /etc/hosts 2023-08-08T22:14:02.696 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hostname 2023-08-08T22:14:02.771 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2023-08-08T22:14:02.844 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2023-08-08T22:14:02.902 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2023-08-08T22:14:02.902 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2023-08-08T22:14:02.903 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2023-08-08T22:14:02.903 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2023-08-08T22:14:02.903 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2023-08-08T22:14:02.903 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2023-08-08T22:14:02.903 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-08T22:14:02.903 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2023-08-08T22:14:02.903 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2023-08-08T22:14:02.904 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-08T22:14:02.904 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2023-08-08T22:14:02.904 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-08T22:14:02.904 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-08T22:14:02.904 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-08T22:14:02.905 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2023-08-08T22:14:02.906 DEBUG:teuthology.parallel:result is None 2023-08-08T22:14:05.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:14:06.204 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-08T22:14:06.205 DEBUG:teuthology.orchestra.run.smithi079:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-08T22:14:06.533 INFO:teuthology.provision.fog.smithi079:Node is ready 2023-08-08T22:14:06.533 DEBUG:teuthology.orchestra.run.smithi079:> hostname 2023-08-08T22:14:06.547 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com 2023-08-08T22:14:06.547 DEBUG:teuthology.orchestra.run.smithi079:> grep smithi079.front.sepia.ceph.com /etc/hosts 2023-08-08T22:14:06.602 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079 2023-08-08T22:14:06.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2023-08-08T22:14:06.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hosts 2023-08-08T22:14:06.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/172.21.15.79/172.21.15.79/g' /etc/hosts 2023-08-08T22:14:06.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hostname 2023-08-08T22:14:06.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2023-08-08T22:14:06.954 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2023-08-08T22:14:07.008 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2023-08-08T22:14:07.008 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2023-08-08T22:14:07.009 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2023-08-08T22:14:07.009 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2023-08-08T22:14:07.009 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2023-08-08T22:14:07.009 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2023-08-08T22:14:07.009 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-08T22:14:07.009 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2023-08-08T22:14:07.009 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2023-08-08T22:14:07.010 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-08T22:14:07.010 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2023-08-08T22:14:07.010 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-08T22:14:07.010 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-08T22:14:07.010 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-08T22:14:07.010 INFO:teuthology.provision.fog.smithi079:Deploy complete! 2023-08-08T22:14:07.011 DEBUG:teuthology.parallel:result is None 2023-08-08T22:14:07.182 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2023-08-08T22:14:07.183 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-08T22:14:07.203 INFO:teuthology.lock.ops:Checking smithi079.front.sepia.ceph.com 2023-08-08T22:14:07.203 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-08T22:14:07.221 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2023-08-08T22:14:07.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-08-08T22:14:07.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:14:07.323 DEBUG:teuthology.orchestra.run.smithi050:> true 2023-08-08T22:14:07.391 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-08-08T22:14:07.391 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2023-08-08T22:14:07.449 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2023-08-08T22:14:07.449 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2023-08-08T22:14:07.509 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2023-08-08T22:14:07.510 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2023-08-08T22:14:07.510 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2023-08-08T22:14:07.510 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2023-08-08T22:14:07.510 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2023-08-08T22:14:07.510 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2023-08-08T22:14:07.510 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-08T22:14:07.510 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2023-08-08T22:14:07.511 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2023-08-08T22:14:07.511 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-08T22:14:07.511 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2023-08-08T22:14:07.511 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-08T22:14:07.511 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-08T22:14:07.511 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-08T22:14:07.512 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-08-08T22:14:07.537 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: set os type and version on server 2023-08-08T22:14:07.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-08-08T22:14:07.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T22:14:07.618 DEBUG:teuthology.orchestra.run.smithi079:> true 2023-08-08T22:14:07.682 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-08-08T22:14:07.683 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2023-08-08T22:14:07.737 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2023-08-08T22:14:07.737 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2023-08-08T22:14:07.792 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2023-08-08T22:14:07.792 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2023-08-08T22:14:07.792 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2023-08-08T22:14:07.792 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2023-08-08T22:14:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2023-08-08T22:14:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2023-08-08T22:14:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-08T22:14:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2023-08-08T22:14:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2023-08-08T22:14:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-08T22:14:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2023-08-08T22:14:07.794 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-08T22:14:07.794 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-08T22:14:07.794 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-08T22:14:07.794 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2023-08-08T22:14:07.820 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:14:07.908 INFO:teuthology.dispatcher.supervisor:Running job 7363440 2023-08-08T22:14:07.909 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-08-08_20:08:30-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7363440 --name rishabh-2023-08-08_20:08:30-fs-wip-rishabh-2023Aug1-3-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} -- /home/teuthworker/archive/rishabh-2023-08-08_20:08:30-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7363440/orig.config.yaml 2023-08-08T22:14:07.912 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-08-08_20:08:30-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7363440 2023-08-08T22:14:07.913 INFO:teuthology.dispatcher.supervisor:Job PID: 24191 2023-08-08T22:14:07.913 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-08T22:16:08.012 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:18:08.182 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:20:08.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:22:08.521 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:24:08.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:26:08.861 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:28:09.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:30:09.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:32:09.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:34:09.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:36:09.696 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:38:09.857 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:40:10.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:42:10.189 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:44:10.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:46:10.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:48:10.673 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:50:10.841 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:52:11.003 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:54:11.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:56:11.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T22:58:11.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T23:00:11.673 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T23:02:11.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T23:02:11.902 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-08T23:02:12.029 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-08T23:02:12.029 INFO:teuthology.nuke:Checking targets against current locks 2023-08-08T23:02:12.077 DEBUG:teuthology.nuke:shortname: smithi050 2023-08-08T23:02:12.077 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-08T23:02:12.098 DEBUG:teuthology.nuke:shortname: smithi079 2023-08-08T23:02:12.098 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-08T23:02:12.101 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-08_20:08:30-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7363440', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-08 22:09:24.428521', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMxtkXLi2QZZ49OlIGSYDRJ9aLySFCbmst91d6QeJVb8RRFBT4fHvSp7WxU0mabm3Y2rc1/FFfMUTkidsQ+GlKA='} 2023-08-08T23:02:12.122 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-08_20:08:30-fs-wip-rishabh-2023Aug1-3-testing-default-smithi/7363440', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-08 22:09:24.430184', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOWqWYDSU6INRo/fYmnacbKZ2qGhezC3nED180H1TZVVcKqkFkDaF2Of3Fjv4rmTlrU+8bFuyakEkPynmBuGN4c='} 2023-08-08T23:02:12.147 INFO:teuthology.orchestra.console.smithi050:Power off 2023-08-08T23:02:12.148 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T23:02:12.170 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2023-08-08T23:02:12.171 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T23:02:12.187 INFO:teuthology.orchestra.console.smithi079:Power off 2023-08-08T23:02:12.188 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T23:02:12.196 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2023-08-08T23:02:12.207 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2023-08-08T23:02:12.207 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T23:02:12.226 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-08-08T23:02:12.297 INFO:teuthology.orchestra.console.smithi050:Power off completed 2023-08-08T23:02:12.328 INFO:teuthology.orchestra.console.smithi079:Power off completed 2023-08-08T23:02:12.453 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com 2023-08-08T23:02:12.476 INFO:teuthology.lock.ops:unlocked: smithi079.front.sepia.ceph.com