2023-10-19T04:39:59.609 INFO:root:teuthology version: 0.0.1.dev201+g202b180
2023-10-19T04:39:59.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T04:39:59.752 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f3ca4a61a60>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f3ca47ffb80>, signals=[15])
2023-10-19T04:39:59.754 INFO:teuthology.lock.ops:Start node 'smithi059.front.sepia.ceph.com' reimaging
2023-10-19T04:39:59.754 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: reset os type and version on server
2023-10-19T04:39:59.755 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server
2023-10-19T04:39:59.782 INFO:teuthology.lock.ops:Start node 'smithi190.front.sepia.ceph.com' reimaging
2023-10-19T04:39:59.782 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: reset os type and version on server
2023-10-19T04:39:59.783 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server
2023-10-19T04:39:59.784 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /host {"name": "smithi059"}
2023-10-19T04:39:59.804 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /host {"name": "smithi190"}
2023-10-19T04:39:59.858 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-10-19T04:39:59.873 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-10-19T04:39:59.901 DEBUG:teuthology.provision.fog.smithi059:Requesting image smithi_centos_9.stream (ID 10)
2023-10-19T04:39:59.902 DEBUG:teuthology.provision.fog.smithi059:FOG request: PUT /host/164 {"imageID": 10}
2023-10-19T04:39:59.918 DEBUG:teuthology.provision.fog.smithi190:Requesting image smithi_centos_9.stream (ID 10)
2023-10-19T04:39:59.918 DEBUG:teuthology.provision.fog.smithi190:FOG request: PUT /host/295 {"imageID": 10}
2023-10-19T04:39:59.945 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of centos 9.stream
2023-10-19T04:39:59.946 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:39:59.961 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of centos 9.stream
2023-10-19T04:39:59.962 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:39:59.988 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /tasktype {"name": "deploy"}
2023-10-19T04:40:00.007 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /tasktype {"name": "deploy"}
2023-10-19T04:40:00.027 DEBUG:teuthology.provision.fog.smithi059:FOG request: POST /host/164/task {"taskTypeID": 1}
2023-10-19T04:40:00.049 DEBUG:teuthology.provision.fog.smithi190:FOG request: POST /host/295/task {"taskTypeID": 1}
2023-10-19T04:40:00.156 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:40:00.169 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:40:00.242 INFO:teuthology.orchestra.console.smithi059:Power off
2023-10-19T04:40:00.243 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-19T04:40:00.257 INFO:teuthology.orchestra.console.smithi190:Power off
2023-10-19T04:40:00.258 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-19T04:40:00.266 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off
2023-10-19T04:40:00.267 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T04:40:00.279 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off
2023-10-19T04:40:00.279 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T04:40:00.293 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off
2023-10-19T04:40:00.306 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2023-10-19T04:40:00.395 INFO:teuthology.orchestra.console.smithi059:Power off completed
2023-10-19T04:40:00.407 INFO:teuthology.orchestra.console.smithi190:Power off completed
2023-10-19T04:40:00.496 INFO:teuthology.orchestra.console.smithi059:Power on
2023-10-19T04:40:00.496 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-10-19T04:40:00.509 INFO:teuthology.orchestra.console.smithi190:Power on
2023-10-19T04:40:00.509 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-10-19T04:40:00.520 DEBUG:teuthology.orchestra.console.smithi059:power on output: Chassis Power Control: Up/On
2023-10-19T04:40:00.521 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T04:40:00.530 DEBUG:teuthology.orchestra.console.smithi190:power on output: Chassis Power Control: Up/On
2023-10-19T04:40:00.531 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T04:40:00.546 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off
2023-10-19T04:40:00.552 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2023-10-19T04:40:04.550 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T04:40:04.558 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T04:40:04.661 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off
2023-10-19T04:40:04.667 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2023-10-19T04:40:08.666 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T04:40:08.674 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T04:40:08.784 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on
2023-10-19T04:40:08.885 INFO:teuthology.orchestra.console.smithi190:Power on completed
2023-10-19T04:40:08.893 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on
2023-10-19T04:40:08.986 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish
2023-10-19T04:40:08.986 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:40:08.994 INFO:teuthology.orchestra.console.smithi059:Power on completed
2023-10-19T04:40:09.096 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish
2023-10-19T04:40:09.096 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:40:24.058 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:40:24.154 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:40:39.138 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:40:39.214 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:40:54.210 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:40:54.274 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:41:09.287 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:41:09.341 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:41:24.370 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:41:24.396 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:41:39.450 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:41:39.453 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:41:54.530 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:41:54.532 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:42:09.610 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:42:09.612 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:42:24.687 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:42:24.689 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:42:39.766 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-10-19T04:42:39.768 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None
2023-10-19T04:42:39.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:42:39.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:42:58.303 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59
2023-10-19T04:42:58.304 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-10-19T04:43:04.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:43:04.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:43:07.386 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-10-19T04:43:07.387 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59
2023-10-19T04:43:13.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:43:13.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:43:16.442 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-10-19T04:43:16.474 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59
2023-10-19T04:43:22.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:43:22.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:43:25.498 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-10-19T04:43:25.534 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59
2023-10-19T04:43:31.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:43:31.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:44:31.567 WARNING:teuthology.provision.fog.smithi190:timed out
2023-10-19T04:44:31.568 WARNING:teuthology.provision.fog.smithi059:timed out
2023-10-19T04:44:37.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:44:37.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:44:37.814 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-19T04:44:37.815 DEBUG:teuthology.orchestra.run.smithi190:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-19T04:44:37.821 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-19T04:44:37.821 DEBUG:teuthology.orchestra.run.smithi059:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-19T04:44:38.147 INFO:teuthology.provision.fog.smithi190:Node is ready
2023-10-19T04:44:38.147 DEBUG:teuthology.orchestra.run.smithi190:> hostname
2023-10-19T04:44:38.154 INFO:teuthology.provision.fog.smithi059:Node is ready
2023-10-19T04:44:38.154 DEBUG:teuthology.orchestra.run.smithi059:> hostname
2023-10-19T04:44:38.163 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190.front.sepia.ceph.com
2023-10-19T04:44:38.163 DEBUG:teuthology.orchestra.run.smithi190:> grep smithi190.front.sepia.ceph.com /etc/hosts
2023-10-19T04:44:38.170 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com
2023-10-19T04:44:38.170 DEBUG:teuthology.orchestra.run.smithi059:> grep smithi059.front.sepia.ceph.com /etc/hosts
2023-10-19T04:44:38.219 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.15.190 smithi190.front.sepia.ceph.com smithi190
2023-10-19T04:44:38.220 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname smithi190
2023-10-19T04:44:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059
2023-10-19T04:44:38.228 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059
2023-10-19T04:44:38.315 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/smithi190.front.sepia.ceph.com/smithi190/g' /etc/hosts
2023-10-19T04:44:38.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hosts
2023-10-19T04:44:38.344 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/172.21.15.190/172.21.15.190/g' /etc/hosts
2023-10-19T04:44:38.375 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/172.21.15.59/172.21.15.59/g' /etc/hosts
2023-10-19T04:44:38.412 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/smithi190.front.sepia.ceph.com/smithi190/g' /etc/hostname
2023-10-19T04:44:38.446 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hostname
2023-10-19T04:44:38.479 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname smithi190
2023-10-19T04:44:38.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059
2023-10-19T04:44:38.547 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release
2023-10-19T04:44:38.587 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release
2023-10-19T04:44:38.601 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream"
2023-10-19T04:44:38.602 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9"
2023-10-19T04:44:38.602 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos"
2023-10-19T04:44:38.602 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora"
2023-10-19T04:44:38.602 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9"
2023-10-19T04:44:38.603 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9"
2023-10-19T04:44:38.603 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9"
2023-10-19T04:44:38.603 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31"
2023-10-19T04:44:38.603 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon"
2023-10-19T04:44:38.603 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-10-19T04:44:38.604 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/"
2023-10-19T04:44:38.604 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-19T04:44:38.604 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-10-19T04:44:38.604 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-19T04:44:38.605 INFO:teuthology.provision.fog.smithi190:Deploy complete!
2023-10-19T04:44:38.607 DEBUG:teuthology.parallel:result is None
2023-10-19T04:44:38.642 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream"
2023-10-19T04:44:38.642 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9"
2023-10-19T04:44:38.642 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos"
2023-10-19T04:44:38.643 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora"
2023-10-19T04:44:38.643 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9"
2023-10-19T04:44:38.643 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9"
2023-10-19T04:44:38.643 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9"
2023-10-19T04:44:38.643 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31"
2023-10-19T04:44:38.644 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon"
2023-10-19T04:44:38.644 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-10-19T04:44:38.644 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/"
2023-10-19T04:44:38.644 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-19T04:44:38.645 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-10-19T04:44:38.645 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-19T04:44:38.645 INFO:teuthology.provision.fog.smithi059:Deploy complete!
2023-10-19T04:44:38.646 DEBUG:teuthology.parallel:result is None
2023-10-19T04:44:38.813 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com
2023-10-19T04:44:38.813 INFO:teuthology.lock.ops:New key found. Updating...
2023-10-19T04:44:38.837 INFO:teuthology.lock.ops:Checking smithi190.front.sepia.ceph.com
2023-10-19T04:44:38.838 INFO:teuthology.lock.ops:New key found. Updating...
2023-10-19T04:44:38.863 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server
2023-10-19T04:44:38.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com'
2023-10-19T04:44:38.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:44:38.960 DEBUG:teuthology.orchestra.run.smithi059:> true
2023-10-19T04:44:39.024 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com'
2023-10-19T04:44:39.025 DEBUG:teuthology.orchestra.run.smithi059:> uname -m
2023-10-19T04:44:39.081 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64
2023-10-19T04:44:39.082 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release
2023-10-19T04:44:39.137 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream"
2023-10-19T04:44:39.137 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9"
2023-10-19T04:44:39.137 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos"
2023-10-19T04:44:39.138 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora"
2023-10-19T04:44:39.138 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9"
2023-10-19T04:44:39.138 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9"
2023-10-19T04:44:39.138 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9"
2023-10-19T04:44:39.139 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31"
2023-10-19T04:44:39.139 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon"
2023-10-19T04:44:39.139 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-10-19T04:44:39.139 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/"
2023-10-19T04:44:39.139 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-19T04:44:39.140 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-10-19T04:44:39.140 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-19T04:44:39.140 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server
2023-10-19T04:44:39.162 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: set os type and version on server
2023-10-19T04:44:39.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com'
2023-10-19T04:44:39.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-19T04:44:39.244 DEBUG:teuthology.orchestra.run.smithi190:> true
2023-10-19T04:44:39.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com'
2023-10-19T04:44:39.310 DEBUG:teuthology.orchestra.run.smithi190:> uname -m
2023-10-19T04:44:39.364 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64
2023-10-19T04:44:39.364 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release
2023-10-19T04:44:39.419 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream"
2023-10-19T04:44:39.419 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9"
2023-10-19T04:44:39.419 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos"
2023-10-19T04:44:39.420 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora"
2023-10-19T04:44:39.420 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9"
2023-10-19T04:44:39.420 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9"
2023-10-19T04:44:39.420 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9"
2023-10-19T04:44:39.421 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31"
2023-10-19T04:44:39.421 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon"
2023-10-19T04:44:39.421 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-10-19T04:44:39.421 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/"
2023-10-19T04:44:39.421 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-19T04:44:39.422 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-10-19T04:44:39.422 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-19T04:44:39.422 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server
2023-10-19T04:44:39.445 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T04:44:39.565 INFO:teuthology.dispatcher.supervisor:Running job 7432150
2023-10-19T04:44:39.565 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/virtualenv/bin/teuthology -v --owner scheduled_ivancich@teuthology --archive /home/teuthworker/archive/ivancich-2023-10-19_04:33:51-rgw-wip-eric-testing-1-distro-default-smithi/7432150 --name ivancich-2023-10-19_04:33:51-rgw-wip-eric-testing-1-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{centos_latest} tasks/test_multi} -- /home/teuthworker/archive/ivancich-2023-10-19_04:33:51-rgw-wip-eric-testing-1-distro-default-smithi/7432150/orig.config.yaml
2023-10-19T04:44:39.568 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ivancich-2023-10-19_04:33:51-rgw-wip-eric-testing-1-distro-default-smithi/7432150
2023-10-19T04:44:39.569 INFO:teuthology.dispatcher.supervisor:Job PID: 3437381
2023-10-19T04:44:39.570 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-10-19T04:46:39.674 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T04:48:39.903 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T04:50:40.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T04:52:40.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T04:54:40.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T04:56:40.565 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T04:58:40.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:00:40.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:02:41.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:04:41.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:06:41.351 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:08:41.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:10:41.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:12:41.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:14:42.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:16:42.652 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:18:42.820 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:20:42.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:22:43.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:24:43.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:26:43.469 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:28:43.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:30:43.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:32:43.980 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:34:44.143 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:36:44.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:38:44.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:40:44.650 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:42:44.830 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:44:44.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:46:45.160 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:48:45.334 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:50:45.501 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:52:45.666 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:54:45.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:56:45.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T05:58:46.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:00:46.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:02:46.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:04:46.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:06:46.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:08:46.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:10:47.123 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:12:47.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:14:47.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:16:47.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:18:47.787 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:20:47.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:22:48.123 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:24:48.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:26:48.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:28:48.622 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:30:48.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:32:48.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:34:49.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:36:49.288 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:38:49.434 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:40:49.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:42:49.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:44:49.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:46:50.042 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:48:50.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:50:50.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:52:50.568 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:54:50.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:56:50.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T06:58:51.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:00:51.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:02:51.383 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:04:51.532 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:06:51.695 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:08:51.865 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:10:52.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:12:52.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:14:52.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:16:52.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:18:52.682 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:20:52.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:22:52.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:24:53.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:26:53.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-19T07:26:53.447 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-10-19T07:26:53.637 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-10-19T07:26:53.637 INFO:teuthology.nuke:Checking targets against current locks
2023-10-19T07:26:53.689 DEBUG:teuthology.nuke:shortname: smithi059
2023-10-19T07:26:53.690 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-10-19T07:26:53.715 DEBUG:teuthology.nuke:shortname: smithi190
2023-10-19T07:26:53.716 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-10-19T07:26:53.719 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2023-10-19_04:33:51-rgw-wip-eric-testing-1-distro-default-smithi/7432150', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-19 04:39:58.789736', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLbC1zTCvGD+VrmLbLk2b/8MIXPAXQEFqxyQ5CtlHhK/c81ZejtbMU2eIT1Ml50Fazq9nZNulp71MGnXbet2pbY='}
2023-10-19T07:26:53.738 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2023-10-19_04:33:51-rgw-wip-eric-testing-1-distro-default-smithi/7432150', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-19 04:39:58.791148', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBCH1wosxmKIgwM+NeBYCGl/P+BwCurW3pP4gVIsnY9/xg3SEQJg4lKgc9SpV0kZr3Fm5gEXCbFvZTn3RWaYNeY='}
2023-10-19T07:26:53.769 INFO:teuthology.orchestra.console.smithi059:Power off
2023-10-19T07:26:53.769 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-19T07:26:53.783 INFO:teuthology.orchestra.console.smithi190:Power off
2023-10-19T07:26:53.783 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-19T07:26:53.795 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off
2023-10-19T07:26:53.796 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T07:26:53.804 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off
2023-10-19T07:26:53.805 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-19T07:26:53.820 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off
2023-10-19T07:26:53.824 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2023-10-19T07:26:53.922 INFO:teuthology.orchestra.console.smithi059:Power off completed
2023-10-19T07:26:53.925 INFO:teuthology.orchestra.console.smithi190:Power off completed
2023-10-19T07:26:54.078 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com
2023-10-19T07:26:54.087 INFO:teuthology.lock.ops:unlocked: smithi190.front.sepia.ceph.com