2024-01-05T00:43:31.718 INFO:root:teuthology version: 0.0.1.dev228+g01dd171
2024-01-05T00:43:31.718 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T00:43:31.859 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fac1483a2b0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fac15791940>, signals=[15])
2024-01-05T00:43:31.859 INFO:teuthology.lock.ops:Start node 'smithi131.front.sepia.ceph.com' reimaging
2024-01-05T00:43:31.859 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: reset os type and version on server
2024-01-05T00:43:31.860 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server
2024-01-05T00:43:31.880 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging
2024-01-05T00:43:31.880 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server
2024-01-05T00:43:31.880 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2024-01-05T00:43:31.998 DEBUG:teuthology.provision.fog.smithi131:Requesting image smithi_centos_9.stream (ID 10)
2024-01-05T00:43:32.005 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_centos_9.stream (ID 10)
2024-01-05T00:43:32.044 INFO:teuthology.provision.fog.smithi131:Scheduling deploy of centos 9.stream
2024-01-05T00:43:32.052 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 9.stream
2024-01-05T00:43:32.331 INFO:teuthology.orchestra.console.smithi131:Power off
2024-01-05T00:43:32.331 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-05T00:43:32.345 INFO:teuthology.orchestra.console.smithi181:Power off
2024-01-05T00:43:32.345 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-05T00:43:32.355 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off
2024-01-05T00:43:32.355 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T00:43:32.367 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off
2024-01-05T00:43:32.367 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T00:43:32.377 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off
2024-01-05T00:43:32.387 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2024-01-05T00:43:32.477 INFO:teuthology.orchestra.console.smithi131:Power off completed
2024-01-05T00:43:32.488 INFO:teuthology.orchestra.console.smithi181:Power off completed
2024-01-05T00:43:32.578 INFO:teuthology.orchestra.console.smithi131:Power on
2024-01-05T00:43:32.578 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-05T00:43:32.591 INFO:teuthology.orchestra.console.smithi181:Power on
2024-01-05T00:43:32.592 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-05T00:43:32.602 DEBUG:teuthology.orchestra.console.smithi131:power on output: Chassis Power Control: Up/On
2024-01-05T00:43:32.602 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T00:43:32.613 DEBUG:teuthology.orchestra.console.smithi181:power on output: Chassis Power Control: Up/On
2024-01-05T00:43:32.614 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T00:43:32.622 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off
2024-01-05T00:43:32.636 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2024-01-05T00:43:36.624 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T00:43:36.637 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T00:43:36.733 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off
2024-01-05T00:43:36.747 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2024-01-05T00:43:36.847 INFO:teuthology.orchestra.console.smithi181:Power on completed
2024-01-05T00:43:36.948 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish
2024-01-05T00:43:40.736 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T00:43:40.846 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on
2024-01-05T00:43:40.947 INFO:teuthology.orchestra.console.smithi131:Power on completed
2024-01-05T00:43:41.047 INFO:teuthology.provision.fog.smithi131:Waiting for deploy to finish
2024-01-05T00:46:07.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:11.072 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-01-05T00:46:12.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:15.068 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131
2024-01-05T00:46:17.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:20.156 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-01-05T00:46:21.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:24.124 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131
2024-01-05T00:46:26.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:29.212 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-01-05T00:46:30.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:33.180 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131
2024-01-05T00:46:35.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:38.268 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-01-05T00:46:39.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:42.236 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131
2024-01-05T00:46:44.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:47.324 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-01-05T00:46:48.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:51.292 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131
2024-01-05T00:46:53.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:46:56.384 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-01-05T00:46:57.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:47:00.348 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131
2024-01-05T00:47:02.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:47:05.468 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-01-05T00:47:06.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:47:11.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:48:06.387 WARNING:teuthology.provision.fog.smithi131:timed out
2024-01-05T00:48:11.480 WARNING:teuthology.provision.fog.smithi181:timed out
2024-01-05T00:48:12.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:48:12.622 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-05T00:48:12.623 DEBUG:teuthology.orchestra.run.smithi131:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-05T00:48:12.945 INFO:teuthology.provision.fog.smithi131:Node is ready
2024-01-05T00:48:12.945 DEBUG:teuthology.orchestra.run.smithi131:> hostname
2024-01-05T00:48:12.960 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131.front.sepia.ceph.com
2024-01-05T00:48:12.960 DEBUG:teuthology.orchestra.run.smithi131:> grep smithi131.front.sepia.ceph.com /etc/hosts
2024-01-05T00:48:13.014 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131 smithi131.front.sepia.ceph.com smithi131
2024-01-05T00:48:13.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131
2024-01-05T00:48:13.116 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hosts
2024-01-05T00:48:13.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/172.21.15.131/172.21.15.131/g' /etc/hosts
2024-01-05T00:48:13.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hostname
2024-01-05T00:48:13.277 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131
2024-01-05T00:48:13.343 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release
2024-01-05T00:48:13.397 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream"
2024-01-05T00:48:13.397 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-05T00:48:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-05T00:48:13.400 INFO:teuthology.provision.fog.smithi131:Deploy complete!
2024-01-05T00:48:13.403 DEBUG:teuthology.parallel:result is None
2024-01-05T00:48:17.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:48:17.803 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-05T00:48:17.803 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-05T00:48:18.110 INFO:teuthology.provision.fog.smithi181:Node is ready
2024-01-05T00:48:18.110 DEBUG:teuthology.orchestra.run.smithi181:> hostname
2024-01-05T00:48:18.166 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com
2024-01-05T00:48:18.166 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts
2024-01-05T00:48:18.222 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181
2024-01-05T00:48:18.222 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181
2024-01-05T00:48:18.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts
2024-01-05T00:48:18.382 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts
2024-01-05T00:48:18.450 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname
2024-01-05T00:48:18.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181
2024-01-05T00:48:18.584 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release
2024-01-05T00:48:18.639 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream"
2024-01-05T00:48:18.639 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9"
2024-01-05T00:48:18.639 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos"
2024-01-05T00:48:18.639 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora"
2024-01-05T00:48:18.639 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9"
2024-01-05T00:48:18.639 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9"
2024-01-05T00:48:18.639 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-05T00:48:18.639 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31"
2024-01-05T00:48:18.640 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon"
2024-01-05T00:48:18.640 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-05T00:48:18.640 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/"
2024-01-05T00:48:18.640 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-05T00:48:18.640 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-05T00:48:18.640 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-05T00:48:18.640 INFO:teuthology.provision.fog.smithi181:Deploy complete!
2024-01-05T00:48:18.641 DEBUG:teuthology.parallel:result is None
2024-01-05T00:48:18.838 INFO:teuthology.lock.ops:Checking smithi131.front.sepia.ceph.com
2024-01-05T00:48:18.838 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-05T00:48:18.864 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com
2024-01-05T00:48:18.864 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-05T00:48:18.883 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: set os type and version on server
2024-01-05T00:48:18.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com'
2024-01-05T00:48:18.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:48:18.966 DEBUG:teuthology.orchestra.run.smithi131:> true
2024-01-05T00:48:19.036 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com'
2024-01-05T00:48:19.036 DEBUG:teuthology.orchestra.run.smithi131:> uname -m
2024-01-05T00:48:19.091 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64
2024-01-05T00:48:19.091 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release
2024-01-05T00:48:19.147 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream"
2024-01-05T00:48:19.147 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9"
2024-01-05T00:48:19.147 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos"
2024-01-05T00:48:19.147 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora"
2024-01-05T00:48:19.147 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9"
2024-01-05T00:48:19.147 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9"
2024-01-05T00:48:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-05T00:48:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31"
2024-01-05T00:48:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon"
2024-01-05T00:48:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-05T00:48:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/"
2024-01-05T00:48:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-05T00:48:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-05T00:48:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-05T00:48:19.148 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server
2024-01-05T00:48:19.172 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server
2024-01-05T00:48:19.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2024-01-05T00:48:19.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-05T00:48:19.254 DEBUG:teuthology.orchestra.run.smithi181:> true
2024-01-05T00:48:19.322 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com'
2024-01-05T00:48:19.323 DEBUG:teuthology.orchestra.run.smithi181:> uname -m
2024-01-05T00:48:19.377 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64
2024-01-05T00:48:19.377 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release
2024-01-05T00:48:19.432 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream"
2024-01-05T00:48:19.432 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9"
2024-01-05T00:48:19.432 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos"
2024-01-05T00:48:19.432 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora"
2024-01-05T00:48:19.432 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9"
2024-01-05T00:48:19.432 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9"
2024-01-05T00:48:19.432 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-05T00:48:19.433 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31"
2024-01-05T00:48:19.433 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon"
2024-01-05T00:48:19.433 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-05T00:48:19.433 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/"
2024-01-05T00:48:19.433 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-05T00:48:19.433 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-05T00:48:19.433 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-05T00:48:19.433 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2024-01-05T00:48:19.456 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T00:48:19.544 INFO:teuthology.dispatcher.supervisor:Running job 7507462
2024-01-05T00:48:19.544 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2024-01-04_20:15:15-rgw-wip-cbodley-testing-distro-default-smithi/7507462 --name cbodley-2024-01-04_20:15:15-rgw-wip-cbodley-testing-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{centos_latest} tasks/test_multi} -- /home/teuthworker/archive/cbodley-2024-01-04_20:15:15-rgw-wip-cbodley-testing-distro-default-smithi/7507462/orig.config.yaml
2024-01-05T00:48:19.547 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2024-01-04_20:15:15-rgw-wip-cbodley-testing-distro-default-smithi/7507462
2024-01-05T00:48:19.547 INFO:teuthology.dispatcher.supervisor:Job PID: 1900992
2024-01-05T00:48:19.548 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-05T00:50:19.648 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T00:52:19.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T00:54:19.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T00:56:20.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T00:58:20.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:00:20.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:02:20.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:04:21.072 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:06:21.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:08:21.416 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:10:21.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:12:21.780 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:14:21.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:16:22.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:18:22.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:20:22.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:22:22.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:24:22.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:26:23.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:28:23.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:30:23.368 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:32:23.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:34:23.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:36:23.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:38:24.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:40:24.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:42:24.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:44:24.560 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:46:24.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:48:24.900 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:50:25.077 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:52:25.240 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:54:25.409 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:56:25.582 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T01:58:25.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:00:25.920 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:02:26.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:04:26.240 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:06:26.414 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:08:26.584 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:10:26.760 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:12:26.928 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:14:27.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:16:27.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:18:27.436 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:20:27.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:22:27.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:24:27.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:26:28.105 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:28:28.264 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:30:28.441 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:32:28.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:34:28.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:36:28.902 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:38:29.072 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:40:29.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:42:29.396 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:44:29.565 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:46:29.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:48:29.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:50:30.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:52:30.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:54:30.412 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:56:30.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T02:58:30.740 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:00:30.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:02:31.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:04:31.244 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:06:31.412 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:08:31.580 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:10:31.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:12:31.913 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:14:32.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:16:32.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:18:32.432 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:20:32.601 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:22:32.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:24:32.940 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:26:33.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:28:33.280 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:30:33.452 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:32:33.656 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:34:33.823 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:36:33.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:38:34.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:40:34.328 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:42:34.493 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:44:34.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-05T03:44:34.842 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-05T03:44:35.036 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-05T03:44:35.036 INFO:teuthology.nuke:Checking targets against current locks
2024-01-05T03:44:35.086 DEBUG:teuthology.nuke:shortname: smithi131
2024-01-05T03:44:35.086 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-05T03:44:35.113 DEBUG:teuthology.nuke:shortname: smithi181
2024-01-05T03:44:35.113 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-05T03:44:35.116 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-01-04_20:15:15-rgw-wip-cbodley-testing-distro-default-smithi/7507462', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-05 00:43:30.743451', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB4i5bF8HfZUOuNHAoIoYx0/Gpna32FLEuQDl6xIk2L8gXvdZLBLq3MWoB5Z1Y72oymfRcVZP8Jiv+9cJTZK/mw='}
2024-01-05T03:44:35.137 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-01-04_20:15:15-rgw-wip-cbodley-testing-distro-default-smithi/7507462', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-05 00:43:30.741707', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHzr3Gi0G8BThSXUX0duuv55A/QD3XucE1dgZWlNbSOxw1SuEk+lmNS+rvpw6vpN1Hnm8rji0h4en3Gd9ucJMfw='}
2024-01-05T03:44:35.167 INFO:teuthology.orchestra.console.smithi131:Power off
2024-01-05T03:44:35.167 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-05T03:44:35.188 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off
2024-01-05T03:44:35.188 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T03:44:35.206 INFO:teuthology.orchestra.console.smithi181:Power off
2024-01-05T03:44:35.207 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-05T03:44:35.216 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off
2024-01-05T03:44:35.227 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off
2024-01-05T03:44:35.227 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-05T03:44:35.248 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2024-01-05T03:44:35.317 INFO:teuthology.orchestra.console.smithi131:Power off completed
2024-01-05T03:44:35.349 INFO:teuthology.orchestra.console.smithi181:Power off completed
2024-01-05T03:44:35.476 INFO:teuthology.lock.ops:unlocked: smithi131.front.sepia.ceph.com
2024-01-05T03:44:35.498 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com