2023-11-17T03:10:35.032 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-17T03:10:35.033 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:10:35.211 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fbf0b93d910>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fbf0c440670>, signals=[15])
2023-11-17T03:10:35.212 INFO:teuthology.lock.ops:Start node 'smithi157.front.sepia.ceph.com' reimaging
2023-11-17T03:10:35.213 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: reset os type and version on server
2023-11-17T03:10:35.213 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server
2023-11-17T03:10:35.236 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging
2023-11-17T03:10:35.236 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server
2023-11-17T03:10:35.236 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2023-11-17T03:10:35.237 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /host {"name": "smithi157"}
2023-11-17T03:10:35.261 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /host {"name": "smithi169"}
2023-11-17T03:10:35.329 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-11-17T03:10:35.343 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-11-17T03:10:35.370 DEBUG:teuthology.provision.fog.smithi157:Requesting image smithi_centos_8.stream (ID 3)
2023-11-17T03:10:35.370 DEBUG:teuthology.provision.fog.smithi157:FOG request: PUT /host/262 {"imageID": 3}
2023-11-17T03:10:35.388 DEBUG:teuthology.provision.fog.smithi169:Requesting image smithi_centos_8.stream (ID 3)
2023-11-17T03:10:35.388 DEBUG:teuthology.provision.fog.smithi169:FOG request: PUT /host/274 {"imageID": 3}
2023-11-17T03:10:35.429 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of centos 8.stream
2023-11-17T03:10:35.429 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:10:35.448 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of centos 8.stream
2023-11-17T03:10:35.449 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:10:35.490 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /tasktype {"name": "deploy"}
2023-11-17T03:10:35.503 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /tasktype {"name": "deploy"}
2023-11-17T03:10:35.540 DEBUG:teuthology.provision.fog.smithi169:FOG request: POST /host/274/task {"taskTypeID": 1}
2023-11-17T03:10:35.546 DEBUG:teuthology.provision.fog.smithi157:FOG request: POST /host/262/task {"taskTypeID": 1}
2023-11-17T03:10:35.602 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:10:35.610 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:10:35.716 INFO:teuthology.orchestra.console.smithi169:Power off
2023-11-17T03:10:35.717 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-17T03:10:35.740 INFO:teuthology.orchestra.console.smithi157:Power off
2023-11-17T03:10:35.740 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-17T03:10:35.751 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off
2023-11-17T03:10:35.751 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:35.772 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off
2023-11-17T03:10:35.773 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:35.792 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on
2023-11-17T03:10:35.804 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off
2023-11-17T03:10:35.905 INFO:teuthology.orchestra.console.smithi157:Power off completed
2023-11-17T03:10:36.005 INFO:teuthology.orchestra.console.smithi157:Power on
2023-11-17T03:10:36.006 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-17T03:10:36.029 DEBUG:teuthology.orchestra.console.smithi157:power on output: Chassis Power Control: Up/On
2023-11-17T03:10:36.029 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:36.060 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off
2023-11-17T03:10:39.794 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:40.734 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:40.836 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on
2023-11-17T03:10:40.842 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on
2023-11-17T03:10:40.943 INFO:teuthology.orchestra.console.smithi157:Power on completed
2023-11-17T03:10:41.044 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish
2023-11-17T03:10:41.044 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:10:44.838 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:44.959 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-11-17T03:10:45.060 INFO:teuthology.orchestra.console.smithi169:Power off completed
2023-11-17T03:10:45.160 INFO:teuthology.orchestra.console.smithi169:Power on
2023-11-17T03:10:45.161 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-17T03:10:45.189 DEBUG:teuthology.orchestra.console.smithi169:power on output: Chassis Power Control: Up/On
2023-11-17T03:10:45.190 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:45.224 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-11-17T03:10:49.226 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:49.338 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-11-17T03:10:53.342 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T03:10:53.458 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on
2023-11-17T03:10:53.558 INFO:teuthology.orchestra.console.smithi169:Power on completed
2023-11-17T03:10:53.659 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish
2023-11-17T03:10:53.659 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:10:56.112 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:11:53.516 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:12:08.594 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:12:08.595 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:12:23.662 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:12:23.663 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:12:38.726 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:12:47.064 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:13:02.150 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:13:02.152 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:13:17.222 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:13:17.223 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:13:32.286 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:13:32.287 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:13:47.350 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-17T03:13:47.351 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-11-17T03:13:47.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:13:47.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:13:50.459 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-17T03:13:50.462 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2023-11-17T03:13:56.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:14:15.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:14:18.106 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-17T03:14:18.106 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2023-11-17T03:14:24.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:14:24.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:14:27.162 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-17T03:14:27.194 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2023-11-17T03:14:33.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:14:33.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:14:36.221 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-17T03:14:36.250 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2023-11-17T03:14:42.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:14:42.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:14:45.306 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-17T03:14:45.307 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169
2023-11-17T03:14:51.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:15:28.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:15:28.803 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-17T03:15:28.804 DEBUG:teuthology.orchestra.run.smithi169:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-17T03:15:31.642 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-17T03:15:37.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:15:40.700 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-17T03:15:46.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:15:57.317 INFO:teuthology.provision.fog.smithi169:Node is ready
2023-11-17T03:15:57.318 DEBUG:teuthology.orchestra.run.smithi169:> hostname
2023-11-17T03:15:57.343 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com
2023-11-17T03:15:57.343 DEBUG:teuthology.orchestra.run.smithi169:> grep smithi169.front.sepia.ceph.com /etc/hosts
2023-11-17T03:15:57.404 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169
2023-11-17T03:15:57.404 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169
2023-11-17T03:15:57.531 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hosts
2023-11-17T03:15:57.563 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/172.21.15.169/172.21.15.169/g' /etc/hosts
2023-11-17T03:15:57.578 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-17T03:15:57.579 DEBUG:teuthology.orchestra.run.smithi157:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-17T03:15:57.633 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hostname
2023-11-17T03:15:57.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169
2023-11-17T03:15:57.771 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release
2023-11-17T03:15:57.829 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream"
2023-11-17T03:15:57.829 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 8"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-17T03:15:57.830 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-17T03:15:57.831 INFO:teuthology.provision.fog.smithi169:Deploy complete!
2023-11-17T03:15:57.832 DEBUG:teuthology.parallel:result is None
2023-11-17T03:16:07.841 INFO:teuthology.provision.fog.smithi157:Node is ready
2023-11-17T03:16:07.841 DEBUG:teuthology.orchestra.run.smithi157:> hostname
2023-11-17T03:16:07.906 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com
2023-11-17T03:16:07.906 DEBUG:teuthology.orchestra.run.smithi157:> grep smithi157.front.sepia.ceph.com /etc/hosts
2023-11-17T03:16:07.967 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157
2023-11-17T03:16:07.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157
2023-11-17T03:16:08.088 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hosts
2023-11-17T03:16:08.120 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/172.21.15.157/172.21.15.157/g' /etc/hosts
2023-11-17T03:16:08.189 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hostname
2023-11-17T03:16:08.258 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157
2023-11-17T03:16:08.326 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream"
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8"
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos"
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora"
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8"
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8"
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 8"
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31"
2023-11-17T03:16:08.385 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-11-17T03:16:08.386 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/"
2023-11-17T03:16:08.386 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-17T03:16:08.386 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-17T03:16:08.386 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-17T03:16:08.386 INFO:teuthology.provision.fog.smithi157:Deploy complete!
2023-11-17T03:16:08.387 DEBUG:teuthology.parallel:result is None
2023-11-17T03:16:08.544 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com
2023-11-17T03:16:08.545 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-17T03:16:08.569 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com
2023-11-17T03:16:08.569 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-17T03:16:08.587 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server
2023-11-17T03:16:08.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com'
2023-11-17T03:16:08.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:16:08.656 DEBUG:teuthology.orchestra.run.smithi157:> true
2023-11-17T03:16:08.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com'
2023-11-17T03:16:08.728 DEBUG:teuthology.orchestra.run.smithi157:> uname -m
2023-11-17T03:16:08.785 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64
2023-11-17T03:16:08.786 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release
2023-11-17T03:16:08.843 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 8"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/"
2023-11-17T03:16:08.844 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-17T03:16:08.845 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-17T03:16:08.845 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-17T03:16:08.845 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server
2023-11-17T03:16:08.869 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server
2023-11-17T03:16:08.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2023-11-17T03:16:08.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-17T03:16:08.981 DEBUG:teuthology.orchestra.run.smithi169:> true
2023-11-17T03:16:09.123 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com'
2023-11-17T03:16:09.123 DEBUG:teuthology.orchestra.run.smithi169:> uname -m
2023-11-17T03:16:09.183 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64
2023-11-17T03:16:09.183 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release
2023-11-17T03:16:09.243 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream"
2023-11-17T03:16:09.243 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8"
2023-11-17T03:16:09.243 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos"
2023-11-17T03:16:09.243 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora"
2023-11-17T03:16:09.243 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8"
2023-11-17T03:16:09.244 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8"
2023-11-17T03:16:09.244 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 8"
2023-11-17T03:16:09.244 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2023-11-17T03:16:09.244 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-11-17T03:16:09.244 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/"
2023-11-17T03:16:09.244 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-17T03:16:09.244 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-17T03:16:09.244 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-17T03:16:09.245 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2023-11-17T03:16:09.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:16:09.401 INFO:teuthology.dispatcher.supervisor:Running job 7461250
2023-11-17T03:16:09.402 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-16_22:27:54-rados-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7461250 --name yuriw-2023-11-16_22:27:54-rados-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} -- /home/teuthworker/archive/yuriw-2023-11-16_22:27:54-rados-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7461250/orig.config.yaml
2023-11-17T03:16:09.404 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-16_22:27:54-rados-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7461250
2023-11-17T03:16:09.405 INFO:teuthology.dispatcher.supervisor:Job PID: 3860783
2023-11-17T03:16:09.405 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-17T03:18:09.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:21:18.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:23:18.978 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:25:19.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:27:19.318 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:29:19.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:32:19.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:34:19.227 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:36:19.341 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:38:26.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:40:26.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:42:26.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:44:36.935 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:44:37.013 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/yuriw-2023-11-16_22:27:54-rados-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-11-17T03:46:38.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:48:38.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:50:51.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:52:51.584 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:54:51.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:56:51.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T03:58:57.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T04:00:57.538 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T04:03:25.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T04:05:25.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T04:07:27.127 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T04:09:31.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T04:11:54.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T04:13:54.674 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-17T04:13:54.745 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-11-17T04:13:54.877 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-17T04:13:54.878 INFO:teuthology.nuke:Checking targets against current locks
2023-11-17T04:13:54.919 DEBUG:teuthology.nuke:shortname: smithi157
2023-11-17T04:13:54.920 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-17T04:13:54.940 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-16_22:27:54-rados-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7461250', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-17 03:10:34.113600', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLFp/FNJZTHF04nScYx7/tWyoJ+gfx57xh60c0U4IQdF8NAgPAnpj+JEqYDhmfxl25i8NUabig5NYPA72FM8V7o='}
2023-11-17T04:13:54.944 DEBUG:teuthology.nuke:shortname: smithi169
2023-11-17T04:13:54.944 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-17T04:13:54.962 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-16_22:27:54-rados-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7461250', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-17 03:10:34.112293', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD4LuIcXgo6b2HzkLp2nRqNFZdfAh/YAXDKQ7i6wBWreE+xc4igA6C6dOFzJA0WsC/DIyF5UXXJSgAYX1rew7rQ='}
2023-11-17T04:13:54.995 INFO:teuthology.orchestra.console.smithi157:Power off
2023-11-17T04:13:54.995 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-17T04:13:55.023 INFO:teuthology.orchestra.console.smithi169:Power off
2023-11-17T04:13:55.023 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-17T04:13:55.034 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off
2023-11-17T04:13:55.034 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T04:13:55.046 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off
2023-11-17T04:13:55.047 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-17T04:13:55.061 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off
2023-11-17T04:13:55.075 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-11-17T04:13:55.162 INFO:teuthology.orchestra.console.smithi157:Power off completed
2023-11-17T04:13:55.176 INFO:teuthology.orchestra.console.smithi169:Power off completed
2023-11-17T04:13:55.312 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com
2023-11-17T04:13:55.328 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com