2023-10-10T06:24:09.250 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-10-10T06:24:09.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:24:09.384 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f44b50c1a90>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f44b4e65b80>, signals=[15])
2023-10-10T06:24:09.385 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging
2023-10-10T06:24:09.386 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server
2023-10-10T06:24:09.386 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2023-10-10T06:24:09.409 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging
2023-10-10T06:24:09.410 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server
2023-10-10T06:24:09.410 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2023-10-10T06:24:09.411 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /host {"name": "smithi027"}
2023-10-10T06:24:09.433 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /host {"name": "smithi050"}
2023-10-10T06:24:09.498 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-10-10T06:24:09.510 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-10-10T06:24:09.546 DEBUG:teuthology.provision.fog.smithi027:Requesting image smithi_centos_8.stream (ID 3)
2023-10-10T06:24:09.546 DEBUG:teuthology.provision.fog.smithi027:FOG request: PUT /host/132 {"imageID": 3}
2023-10-10T06:24:09.551 DEBUG:teuthology.provision.fog.smithi050:Requesting image smithi_centos_8.stream (ID 3)
2023-10-10T06:24:09.551 DEBUG:teuthology.provision.fog.smithi050:FOG request: PUT /host/155 {"imageID": 3}
2023-10-10T06:24:09.588 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 8.stream
2023-10-10T06:24:09.588 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:24:09.591 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 8.stream
2023-10-10T06:24:09.591 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:24:09.651 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /tasktype {"name": "deploy"}
2023-10-10T06:24:09.655 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /tasktype {"name": "deploy"}
2023-10-10T06:24:09.695 DEBUG:teuthology.provision.fog.smithi050:FOG request: POST /host/155/task {"taskTypeID": 1}
2023-10-10T06:24:09.698 DEBUG:teuthology.provision.fog.smithi027:FOG request: POST /host/132/task {"taskTypeID": 1}
2023-10-10T06:24:09.748 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:24:09.759 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:24:09.848 INFO:teuthology.orchestra.console.smithi050:Power off
2023-10-10T06:24:09.849 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-10T06:24:09.862 INFO:teuthology.orchestra.console.smithi027:Power off
2023-10-10T06:24:09.862 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-10T06:24:09.874 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off
2023-10-10T06:24:09.874 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:09.884 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off
2023-10-10T06:24:09.884 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:09.897 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2023-10-10T06:24:09.905 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on
2023-10-10T06:24:13.898 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:13.906 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:14.008 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2023-10-10T06:24:14.015 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on
2023-10-10T06:24:18.014 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:18.021 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:18.123 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2023-10-10T06:24:18.130 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2023-10-10T06:24:18.225 INFO:teuthology.orchestra.console.smithi050:Power off completed
2023-10-10T06:24:18.232 INFO:teuthology.orchestra.console.smithi027:Power off completed
2023-10-10T06:24:18.326 INFO:teuthology.orchestra.console.smithi050:Power on
2023-10-10T06:24:18.327 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-10-10T06:24:18.335 INFO:teuthology.orchestra.console.smithi027:Power on
2023-10-10T06:24:18.335 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-10-10T06:24:18.354 DEBUG:teuthology.orchestra.console.smithi050:power on output: Chassis Power Control: Up/On
2023-10-10T06:24:18.355 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:18.368 DEBUG:teuthology.orchestra.console.smithi027:power on output: Chassis Power Control: Up/On
2023-10-10T06:24:18.369 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:18.378 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2023-10-10T06:24:18.395 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2023-10-10T06:24:22.382 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:22.396 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:22.494 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2023-10-10T06:24:22.504 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2023-10-10T06:24:26.498 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:26.507 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T06:24:26.610 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2023-10-10T06:24:26.618 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on
2023-10-10T06:24:26.711 INFO:teuthology.orchestra.console.smithi050:Power on completed
2023-10-10T06:24:26.720 INFO:teuthology.orchestra.console.smithi027:Power on completed
2023-10-10T06:24:26.812 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish
2023-10-10T06:24:26.813 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:24:26.820 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish
2023-10-10T06:24:26.821 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:24:41.894 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:24:41.896 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:24:56.970 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:24:56.972 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:25:12.058 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:25:12.060 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:25:27.149 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:25:27.150 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:25:42.234 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:25:42.236 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:25:57.317 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:25:57.319 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:26:12.418 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:26:12.420 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:26:27.506 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:26:27.508 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:26:42.602 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:26:42.604 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:26:57.690 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:26:57.692 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None
2023-10-10T06:26:57.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:27:12.794 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:27:16.218 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2023-10-10T06:27:22.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:27:25.274 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2023-10-10T06:27:27.878 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:27:31.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:27:34.334 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2023-10-10T06:27:40.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:27:42.946 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None
2023-10-10T06:27:43.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:27:43.133 ERROR:teuthology.orchestra.connection:Error authenticating with smithi027.front.sepia.ceph.com: Authentication failed.
2023-10-10T06:27:43.135 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-10T06:27:43.135 DEBUG:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-10T06:27:43.136 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-10-10T06:27:43.418 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2023-10-10T06:27:44.144 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-10-10T06:27:49.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:27:52.474 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2023-10-10T06:27:58.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:28:01.530 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2023-10-10T06:28:07.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:28:10.586 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2023-10-10T06:28:16.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:29:16.591 WARNING:teuthology.provision.fog.smithi050:timed out
2023-10-10T06:29:22.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:29:22.824 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-10T06:29:22.824 DEBUG:teuthology.orchestra.run.smithi050:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-10T06:29:23.101 INFO:teuthology.provision.fog.smithi050:Node is ready
2023-10-10T06:29:23.101 DEBUG:teuthology.orchestra.run.smithi050:> hostname
2023-10-10T06:29:23.122 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com
2023-10-10T06:29:23.122 DEBUG:teuthology.orchestra.run.smithi050:> grep smithi050.front.sepia.ceph.com /etc/hosts
2023-10-10T06:29:23.183 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050
2023-10-10T06:29:23.183 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050
2023-10-10T06:29:23.280 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hosts
2023-10-10T06:29:23.311 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/172.21.15.50/172.21.15.50/g' /etc/hosts
2023-10-10T06:29:23.382 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hostname
2023-10-10T06:29:23.453 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050
2023-10-10T06:29:23.522 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release
2023-10-10T06:29:23.581 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream"
2023-10-10T06:29:23.581 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8"
2023-10-10T06:29:23.582 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos"
2023-10-10T06:29:23.582 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora"
2023-10-10T06:29:23.582 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8"
2023-10-10T06:29:23.582 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8"
2023-10-10T06:29:23.583 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 8"
2023-10-10T06:29:23.583 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31"
2023-10-10T06:29:23.583 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-10-10T06:29:23.583 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/"
2023-10-10T06:29:23.583 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-10T06:29:23.584 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-10-10T06:29:23.584 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-10T06:29:23.585 INFO:teuthology.provision.fog.smithi050:Deploy complete!
2023-10-10T06:29:23.586 DEBUG:teuthology.parallel:result is None
2023-10-10T06:30:29.242 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-10-10T06:30:29.244 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi027
2023-10-10T06:30:38.254 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com'
2023-10-10T06:30:38.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:30:38.473 DEBUG:teuthology.orchestra.run.smithi027:> true
2023-10-10T06:30:38.756 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com'
2023-10-10T06:30:38.756 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-10T06:30:38.757 DEBUG:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-10T06:30:38.775 INFO:teuthology.provision.fog.smithi027:Node is ready
2023-10-10T06:30:38.776 DEBUG:teuthology.orchestra.run.smithi027:> hostname
2023-10-10T06:30:38.835 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com
2023-10-10T06:30:38.836 DEBUG:teuthology.orchestra.run.smithi027:> grep smithi027.front.sepia.ceph.com /etc/hosts
2023-10-10T06:30:38.895 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027
2023-10-10T06:30:38.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027
2023-10-10T06:30:38.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hosts
2023-10-10T06:30:39.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/172.21.15.27/172.21.15.27/g' /etc/hosts
2023-10-10T06:30:39.096 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hostname
2023-10-10T06:30:39.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027
2023-10-10T06:30:39.232 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release
2023-10-10T06:30:39.291 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream"
2023-10-10T06:30:39.292 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8"
2023-10-10T06:30:39.292 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos"
2023-10-10T06:30:39.292 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora"
2023-10-10T06:30:39.292 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8"
2023-10-10T06:30:39.292 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8"
2023-10-10T06:30:39.292 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 8"
2023-10-10T06:30:39.293 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31"
2023-10-10T06:30:39.293 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-10-10T06:30:39.293 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/"
2023-10-10T06:30:39.293 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-10T06:30:39.293 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-10-10T06:30:39.294 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-10T06:30:39.294 INFO:teuthology.provision.fog.smithi027:Deploy complete!
2023-10-10T06:30:39.295 DEBUG:teuthology.parallel:result is None
2023-10-10T06:30:39.442 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com
2023-10-10T06:30:39.442 INFO:teuthology.lock.ops:New key found. Updating...
2023-10-10T06:30:39.467 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com
2023-10-10T06:30:39.468 INFO:teuthology.lock.ops:New key found. Updating...
2023-10-10T06:30:39.485 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server
2023-10-10T06:30:39.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com'
2023-10-10T06:30:39.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:30:39.552 DEBUG:teuthology.orchestra.run.smithi027:> true
2023-10-10T06:30:39.620 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com'
2023-10-10T06:30:39.620 DEBUG:teuthology.orchestra.run.smithi027:> uname -m
2023-10-10T06:30:39.678 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64
2023-10-10T06:30:39.678 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release
2023-10-10T06:30:39.736 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream"
2023-10-10T06:30:39.736 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8"
2023-10-10T06:30:39.736 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos"
2023-10-10T06:30:39.736 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora"
2023-10-10T06:30:39.737 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8"
2023-10-10T06:30:39.737 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8"
2023-10-10T06:30:39.737 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 8"
2023-10-10T06:30:39.737 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31"
2023-10-10T06:30:39.737 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-10-10T06:30:39.738 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/"
2023-10-10T06:30:39.738 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-10T06:30:39.738 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-10-10T06:30:39.738 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-10T06:30:39.739 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2023-10-10T06:30:39.759 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server
2023-10-10T06:30:39.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com'
2023-10-10T06:30:39.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-10T06:30:39.828 DEBUG:teuthology.orchestra.run.smithi050:> true
2023-10-10T06:30:39.924 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com'
2023-10-10T06:30:39.925 DEBUG:teuthology.orchestra.run.smithi050:> uname -m
2023-10-10T06:30:39.984 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64
2023-10-10T06:30:39.984 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release
2023-10-10T06:30:40.043 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream"
2023-10-10T06:30:40.044 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8"
2023-10-10T06:30:40.044 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos"
2023-10-10T06:30:40.044 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora"
2023-10-10T06:30:40.044 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8"
2023-10-10T06:30:40.045 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8"
2023-10-10T06:30:40.045 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 8"
2023-10-10T06:30:40.045 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31"
2023-10-10T06:30:40.045 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-10-10T06:30:40.045 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/"
2023-10-10T06:30:40.046 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-10T06:30:40.046 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-10-10T06:30:40.046 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-10T06:30:40.047 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2023-10-10T06:30:40.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:30:40.154 INFO:teuthology.dispatcher.supervisor:Running job 7419389
2023-10-10T06:30:40.154 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_gsalomon@teuthology --archive /home/teuthworker/archive/gsalomon-2023-10-09_23:17:31-rgw-QE_defect_fixes-distro-default-smithi/7419389 --name gsalomon-2023-10-09_23:17:31-rgw-QE_defect_fixes-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi} -- /home/teuthworker/archive/gsalomon-2023-10-09_23:17:31-rgw-QE_defect_fixes-distro-default-smithi/7419389/orig.config.yaml
2023-10-10T06:30:40.157 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gsalomon-2023-10-09_23:17:31-rgw-QE_defect_fixes-distro-default-smithi/7419389
2023-10-10T06:30:40.158 INFO:teuthology.dispatcher.supervisor:Job PID: 2624003
2023-10-10T06:30:40.158 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-10-10T06:32:40.177 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:34:40.346 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:36:40.515 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:38:40.684 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:40:40.845 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:42:41.018 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:44:41.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:46:41.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:48:41.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:50:41.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:52:41.825 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:54:41.989 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:56:42.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T06:58:42.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:00:42.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:02:42.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:04:42.823 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:06:42.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:08:43.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:10:43.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:12:43.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:14:43.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:16:43.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:18:43.941 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:20:44.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:22:44.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:24:44.429 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:26:44.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:28:44.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:30:44.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:32:45.108 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:34:45.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:36:45.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:38:45.607 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:40:45.779 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:42:45.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:44:46.102 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:46:46.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:48:46.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:50:46.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:52:46.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:54:46.970 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:56:47.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T07:58:47.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:00:47.493 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:02:47.675 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:04:47.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:06:48.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:08:48.181 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:10:48.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:12:48.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:14:48.689 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:16:48.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:18:49.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:20:49.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:22:49.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:24:49.565 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:26:49.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:28:49.867 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:30:50.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:32:50.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:34:50.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:36:50.520 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:38:50.684 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:40:50.855 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:42:51.027 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:44:51.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:46:51.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:48:51.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:50:51.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:52:51.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-10T08:52:51.977 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-10-10T08:52:52.158 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-10-10T08:52:52.159 INFO:teuthology.nuke:Checking targets against current locks
2023-10-10T08:52:52.210 DEBUG:teuthology.nuke:shortname: smithi027
2023-10-10T08:52:52.210 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-10-10T08:52:52.234 DEBUG:teuthology.nuke:shortname: smithi050
2023-10-10T08:52:52.235 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-10-10T08:52:52.238 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gsalomon-2023-10-09_23:17:31-rgw-QE_defect_fixes-distro-default-smithi/7419389', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 06:24:08.462994', 'locked_by': 'scheduled_gsalomon@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDdnX/bC87RmNpkG0yQz8rkcMXiUxOyoYpOJrTTj/EhjBIKfocBnAQPzKKK7u2PiimWwX2Clx+F7vntQJkY6kvk='}
2023-10-10T08:52:52.259 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gsalomon-2023-10-09_23:17:31-rgw-QE_defect_fixes-distro-default-smithi/7419389', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 06:24:08.461406', 'locked_by': 'scheduled_gsalomon@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKLZWjtlOiVm5O2M+cfaGgWJV9yNBTflUKvMSTsJAKnGMOubtUupjPSqmijoSOcxqdXDsoxG/kHjTvHW0NHKWuo='}
2023-10-10T08:52:52.289 INFO:teuthology.orchestra.console.smithi027:Power off
2023-10-10T08:52:52.289 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-10T08:52:52.303 INFO:teuthology.orchestra.console.smithi050:Power off
2023-10-10T08:52:52.303 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-10T08:52:52.318 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off
2023-10-10T08:52:52.319 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T08:52:52.330 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off
2023-10-10T08:52:52.330 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-10T08:52:52.347 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2023-10-10T08:52:52.354 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2023-10-10T08:52:52.448 INFO:teuthology.orchestra.console.smithi027:Power off completed
2023-10-10T08:52:52.455 INFO:teuthology.orchestra.console.smithi050:Power off completed
2023-10-10T08:52:52.601 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com
2023-10-10T08:52:52.609 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com