2023-11-14T09:08:47.559 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-14T09:08:47.560 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:08:47.688 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f74de32dc10>, signals=[15]) 2023-11-14T09:08:47.690 INFO:teuthology.lock.ops:Start node 'smithi019.front.sepia.ceph.com' reimaging 2023-11-14T09:08:47.690 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: reset os type and version on server 2023-11-14T09:08:47.690 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2023-11-14T09:08:47.715 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging 2023-11-14T09:08:47.715 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server 2023-11-14T09:08:47.715 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-11-14T09:08:47.717 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /host {"name": "smithi019"} 2023-11-14T09:08:47.740 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /host {"name": "smithi062"} 2023-11-14T09:08:47.797 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-14T09:08:47.807 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-14T09:08:47.843 DEBUG:teuthology.provision.fog.smithi019:Requesting image smithi_centos_9.stream (ID 10) 2023-11-14T09:08:47.843 DEBUG:teuthology.provision.fog.smithi019:FOG request: PUT /host/125 {"imageID": 10} 2023-11-14T09:08:47.849 DEBUG:teuthology.provision.fog.smithi062:Requesting image smithi_centos_9.stream (ID 10) 2023-11-14T09:08:47.849 DEBUG:teuthology.provision.fog.smithi062:FOG request: PUT /host/167 {"imageID": 10} 2023-11-14T09:08:47.893 INFO:teuthology.provision.fog.smithi019:Scheduling deploy of centos 9.stream 2023-11-14T09:08:47.893 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:08:47.897 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of centos 9.stream 2023-11-14T09:08:47.897 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:08:47.963 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /tasktype {"name": "deploy"} 2023-11-14T09:08:47.965 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /tasktype {"name": "deploy"} 2023-11-14T09:08:48.001 DEBUG:teuthology.provision.fog.smithi019:FOG request: POST /host/125/task {"taskTypeID": 1} 2023-11-14T09:08:48.005 DEBUG:teuthology.provision.fog.smithi062:FOG request: POST /host/167/task {"taskTypeID": 1} 2023-11-14T09:08:48.056 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:08:48.059 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:08:48.173 INFO:teuthology.orchestra.console.smithi062:Power off 2023-11-14T09:08:48.174 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T09:08:48.188 INFO:teuthology.orchestra.console.smithi019:Power off 2023-11-14T09:08:48.189 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T09:08:48.199 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2023-11-14T09:08:48.200 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:08:48.211 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2023-11-14T09:08:48.212 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:08:48.225 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-11-14T09:08:48.234 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2023-11-14T09:08:48.334 INFO:teuthology.orchestra.console.smithi019:Power off completed 2023-11-14T09:08:48.435 INFO:teuthology.orchestra.console.smithi019:Power on 2023-11-14T09:08:48.436 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-14T09:08:48.459 DEBUG:teuthology.orchestra.console.smithi019:power on output: Chassis Power Control: Up/On 2023-11-14T09:08:48.459 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:08:48.487 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2023-11-14T09:08:52.229 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:08:52.339 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-11-14T09:08:52.488 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:08:52.596 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2023-11-14T09:08:52.697 INFO:teuthology.orchestra.console.smithi019:Power on completed 2023-11-14T09:08:52.798 INFO:teuthology.provision.fog.smithi019:Waiting for deploy to finish 2023-11-14T09:08:52.799 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:08:56.342 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:08:56.451 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-11-14T09:09:00.454 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:09:00.563 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-11-14T09:09:00.665 INFO:teuthology.orchestra.console.smithi062:Power off completed 2023-11-14T09:09:00.766 INFO:teuthology.orchestra.console.smithi062:Power on 2023-11-14T09:09:00.767 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-14T09:09:00.788 DEBUG:teuthology.orchestra.console.smithi062:power on output: Chassis Power Control: Up/On 2023-11-14T09:09:00.789 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:09:00.811 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-11-14T09:09:04.814 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:09:04.923 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-11-14T09:09:07.877 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:09:08.925 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:09:09.034 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-11-14T09:09:09.135 INFO:teuthology.orchestra.console.smithi062:Power on completed 2023-11-14T09:09:09.236 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish 2023-11-14T09:09:09.237 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:09:23.014 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:09:24.399 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:09:38.202 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:09:39.643 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:09:53.414 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:09:54.848 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:10:08.626 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:10:10.056 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:10:23.846 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:10:25.268 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:10:39.118 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:10:40.484 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:10:54.443 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:10:55.778 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:11:09.798 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:11:11.138 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:11:25.282 DEBUG:teuthology.provision.fog.smithi019:FOG request: GET /task/active None 2023-11-14T09:11:26.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:11:26.548 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:11:42.602 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:11:44.509 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2023-11-14T09:11:50.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:11:53.594 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2023-11-14T09:11:58.450 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-11-14T09:11:59.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:11:59.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:12:02.654 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2023-11-14T09:12:08.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:12:11.738 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2023-11-14T09:12:17.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:12:17.786 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-11-14T09:12:23.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:12:26.846 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-11-14T09:12:32.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:12:35.930 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-11-14T09:12:41.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:12:44.990 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-11-14T09:12:50.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:13:17.771 WARNING:teuthology.provision.fog.smithi019:timed out 2023-11-14T09:13:23.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:13:24.083 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-14T09:13:24.084 DEBUG:teuthology.orchestra.run.smithi019:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-14T09:13:24.408 INFO:teuthology.provision.fog.smithi019:Node is ready 2023-11-14T09:13:24.408 DEBUG:teuthology.orchestra.run.smithi019:> hostname 2023-11-14T09:13:24.423 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019.front.sepia.ceph.com 2023-11-14T09:13:24.424 DEBUG:teuthology.orchestra.run.smithi019:> grep smithi019.front.sepia.ceph.com /etc/hosts 2023-11-14T09:13:24.479 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19 smithi019.front.sepia.ceph.com smithi019 2023-11-14T09:13:24.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname smithi019 2023-11-14T09:13:24.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/smithi019.front.sepia.ceph.com/smithi019/g' /etc/hosts 2023-11-14T09:13:24.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/172.21.15.19/172.21.15.19/g' /etc/hosts 2023-11-14T09:13:24.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/smithi019.front.sepia.ceph.com/smithi019/g' /etc/hostname 2023-11-14T09:13:24.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname smithi019 2023-11-14T09:13:24.828 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2023-11-14T09:13:24.883 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2023-11-14T09:13:24.883 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2023-11-14T09:13:24.883 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2023-11-14T09:13:24.884 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2023-11-14T09:13:24.884 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2023-11-14T09:13:24.884 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2023-11-14T09:13:24.884 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-14T09:13:24.884 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2023-11-14T09:13:24.885 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2023-11-14T09:13:24.885 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-14T09:13:24.885 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2023-11-14T09:13:24.885 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T09:13:24.885 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-14T09:13:24.886 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-14T09:13:24.886 INFO:teuthology.provision.fog.smithi019:Deploy complete! 2023-11-14T09:13:24.887 DEBUG:teuthology.parallel:result is None 2023-11-14T09:13:51.023 WARNING:teuthology.provision.fog.smithi062:timed out 2023-11-14T09:13:57.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:13:57.305 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-14T09:13:57.306 DEBUG:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-14T09:13:57.632 INFO:teuthology.provision.fog.smithi062:Node is ready 2023-11-14T09:13:57.633 DEBUG:teuthology.orchestra.run.smithi062:> hostname 2023-11-14T09:13:57.648 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com 2023-11-14T09:13:57.648 DEBUG:teuthology.orchestra.run.smithi062:> grep smithi062.front.sepia.ceph.com /etc/hosts 2023-11-14T09:13:57.701 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062 2023-11-14T09:13:57.702 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2023-11-14T09:13:57.804 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hosts 2023-11-14T09:13:57.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/172.21.15.62/172.21.15.62/g' /etc/hosts 2023-11-14T09:13:57.904 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hostname 2023-11-14T09:13:57.974 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2023-11-14T09:13:58.042 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-11-14T09:13:58.094 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2023-11-14T09:13:58.095 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2023-11-14T09:13:58.095 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2023-11-14T09:13:58.095 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2023-11-14T09:13:58.095 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2023-11-14T09:13:58.095 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2023-11-14T09:13:58.096 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-14T09:13:58.096 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-11-14T09:13:58.096 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2023-11-14T09:13:58.096 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-14T09:13:58.096 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2023-11-14T09:13:58.097 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T09:13:58.097 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-14T09:13:58.097 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-14T09:13:58.097 INFO:teuthology.provision.fog.smithi062:Deploy complete! 2023-11-14T09:13:58.098 DEBUG:teuthology.parallel:result is None 2023-11-14T09:13:58.282 INFO:teuthology.lock.ops:Checking smithi019.front.sepia.ceph.com 2023-11-14T09:13:58.283 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-14T09:13:58.306 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com 2023-11-14T09:13:58.306 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-14T09:13:58.332 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: set os type and version on server 2023-11-14T09:13:58.332 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-11-14T09:13:58.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:13:58.428 DEBUG:teuthology.orchestra.run.smithi019:> true 2023-11-14T09:13:58.604 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-11-14T09:13:58.605 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2023-11-14T09:13:58.659 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2023-11-14T09:13:58.659 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2023-11-14T09:13:58.713 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2023-11-14T09:13:58.714 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2023-11-14T09:13:58.714 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2023-11-14T09:13:58.714 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2023-11-14T09:13:58.714 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2023-11-14T09:13:58.714 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2023-11-14T09:13:58.715 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-14T09:13:58.715 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2023-11-14T09:13:58.715 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2023-11-14T09:13:58.715 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-14T09:13:58.716 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2023-11-14T09:13:58.716 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T09:13:58.716 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-14T09:13:58.716 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-14T09:13:58.717 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2023-11-14T09:13:58.764 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server 2023-11-14T09:13:58.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-11-14T09:13:58.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T09:13:58.860 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-11-14T09:13:58.934 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-11-14T09:13:58.934 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-11-14T09:13:58.987 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-11-14T09:13:58.988 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-11-14T09:13:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2023-11-14T09:13:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2023-11-14T09:13:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2023-11-14T09:13:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2023-11-14T09:13:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2023-11-14T09:13:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2023-11-14T09:13:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-14T09:13:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-11-14T09:13:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2023-11-14T09:13:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-14T09:13:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2023-11-14T09:13:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T09:13:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-14T09:13:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-14T09:13:59.045 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-11-14T09:13:59.072 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:13:59.163 INFO:teuthology.dispatcher.supervisor:Running job 7456965 2023-11-14T09:13:59.164 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_mkogan@teuthology --archive /home/teuthworker/archive/mkogan-2023-11-14_09:08:23-rgw-rgw-wip-t63445-valg-supp_i001-distro-default-smithi/7456965 --name mkogan-2023-11-14_09:08:23-rgw-rgw-wip-t63445-valg-supp_i001-distro-default-smithi --description rgw/notifications/{beast bluestore-bitmap centos_latest fixed-2 ignore-pg-availability overrides tasks/{0-install test_amqp test_kafka test_others}} -- /home/teuthworker/archive/mkogan-2023-11-14_09:08:23-rgw-rgw-wip-t63445-valg-supp_i001-distro-default-smithi/7456965/orig.config.yaml 2023-11-14T09:13:59.167 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mkogan-2023-11-14_09:08:23-rgw-rgw-wip-t63445-valg-supp_i001-distro-default-smithi/7456965 2023-11-14T09:13:59.168 INFO:teuthology.dispatcher.supervisor:Job PID: 3279438 2023-11-14T09:13:59.168 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-14T09:15:59.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:17:59.448 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:19:59.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:21:59.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:23:59.928 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:26:00.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:28:00.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:30:00.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:32:00.681 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:34:00.855 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:36:01.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:38:01.196 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:40:01.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:42:01.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:44:01.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:46:01.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:48:01.990 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:50:02.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T09:50:02.234 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-14T09:50:02.408 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-14T09:50:02.409 INFO:teuthology.nuke:Checking targets against current locks 2023-11-14T09:50:02.457 DEBUG:teuthology.nuke:shortname: smithi019 2023-11-14T09:50:02.458 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-14T09:50:02.478 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mkogan-2023-11-14_09:08:23-rgw-rgw-wip-t63445-valg-supp_i001-distro-default-smithi/7456965', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 09:08:46.758543', 'locked_by': 'scheduled_mkogan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEecv6l/ffnoeQt+AfnN3kGbroFyqAOG0r9EL4uDmJpeXYmsxD2SYxMgaCd5r1lpVIEzaxEvcuD33FX4u95Q+ao='} 2023-11-14T09:50:02.480 DEBUG:teuthology.nuke:shortname: smithi062 2023-11-14T09:50:02.481 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-14T09:50:02.503 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mkogan-2023-11-14_09:08:23-rgw-rgw-wip-t63445-valg-supp_i001-distro-default-smithi/7456965', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 09:08:46.757132', 'locked_by': 'scheduled_mkogan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDZxj4bEYxwsVvU/wa4svKaR7/r7OpwhsXSER5PEgB73+HKlWFRld1bUAs3YmXln3mBsFk16ESBdiBRrCU3lwps='} 2023-11-14T09:50:02.525 INFO:teuthology.orchestra.console.smithi019:Power off 2023-11-14T09:50:02.526 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T09:50:02.547 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2023-11-14T09:50:02.547 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:50:02.566 INFO:teuthology.orchestra.console.smithi062:Power off 2023-11-14T09:50:02.566 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T09:50:02.574 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2023-11-14T09:50:02.587 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2023-11-14T09:50:02.587 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T09:50:02.607 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-11-14T09:50:02.676 INFO:teuthology.orchestra.console.smithi019:Power off completed 2023-11-14T09:50:02.708 INFO:teuthology.orchestra.console.smithi062:Power off completed 2023-11-14T09:50:02.825 INFO:teuthology.lock.ops:unlocked: smithi019.front.sepia.ceph.com 2023-11-14T09:50:02.850 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com