2023-12-06T09:33:16.740 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-06T09:33:16.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:33:16.888 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f0cc4557e20>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f0cc425ed30>, signals=[15])
2023-12-06T09:33:16.888 INFO:teuthology.lock.ops:Start node 'smithi018.front.sepia.ceph.com' reimaging
2023-12-06T09:33:16.888 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: reset os type and version on server
2023-12-06T09:33:16.888 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server
2023-12-06T09:33:16.907 INFO:teuthology.lock.ops:Start node 'smithi038.front.sepia.ceph.com' reimaging
2023-12-06T09:33:16.907 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: reset os type and version on server
2023-12-06T09:33:16.908 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2023-12-06T09:33:17.040 DEBUG:teuthology.provision.fog.smithi018:Requesting image smithi_centos_9.stream (ID 10)
2023-12-06T09:33:17.048 DEBUG:teuthology.provision.fog.smithi038:Requesting image smithi_centos_9.stream (ID 10)
2023-12-06T09:33:17.100 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of centos 9.stream
2023-12-06T09:33:17.101 INFO:teuthology.provision.fog.smithi018:Scheduling deploy of centos 9.stream
2023-12-06T09:33:17.356 INFO:teuthology.orchestra.console.smithi038:Power off
2023-12-06T09:33:17.357 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-06T09:33:17.372 INFO:teuthology.orchestra.console.smithi018:Power off
2023-12-06T09:33:17.372 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-06T09:33:17.382 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off
2023-12-06T09:33:17.383 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:17.395 DEBUG:teuthology.orchestra.console.smithi018:power off output: Chassis Power Control: Down/Off
2023-12-06T09:33:17.395 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:17.404 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on
2023-12-06T09:33:17.417 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on
2023-12-06T09:33:21.408 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:21.419 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:21.522 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on
2023-12-06T09:33:21.528 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on
2023-12-06T09:33:25.526 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:25.533 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:25.635 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2023-12-06T09:33:25.641 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on
2023-12-06T09:33:25.736 INFO:teuthology.orchestra.console.smithi038:Power off completed
2023-12-06T09:33:25.837 INFO:teuthology.orchestra.console.smithi038:Power on
2023-12-06T09:33:25.838 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-06T09:33:25.859 DEBUG:teuthology.orchestra.console.smithi038:power on output: Chassis Power Control: Up/On
2023-12-06T09:33:25.859 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:25.881 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2023-12-06T09:33:29.645 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:29.754 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is off
2023-12-06T09:33:29.854 INFO:teuthology.orchestra.console.smithi018:Power off completed
2023-12-06T09:33:29.883 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:29.956 INFO:teuthology.orchestra.console.smithi018:Power on
2023-12-06T09:33:29.956 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-06T09:33:29.977 DEBUG:teuthology.orchestra.console.smithi018:power on output: Chassis Power Control: Up/On
2023-12-06T09:33:29.977 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:29.991 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2023-12-06T09:33:30.001 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is off
2023-12-06T09:33:33.994 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:34.002 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:34.111 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is off
2023-12-06T09:33:34.149 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on
2023-12-06T09:33:34.249 INFO:teuthology.orchestra.console.smithi038:Power on completed
2023-12-06T09:33:34.350 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2023-12-06T09:33:38.114 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T09:33:38.223 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on
2023-12-06T09:33:38.324 INFO:teuthology.orchestra.console.smithi018:Power on completed
2023-12-06T09:33:38.424 INFO:teuthology.provision.fog.smithi018:Waiting for deploy to finish
2023-12-06T09:36:05.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:08.386 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-06T09:36:09.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:12.422 WARNING:teuthology.provision.fog.smithi018:[Errno None] Unable to connect to port 22 on 172.21.15.18
2023-12-06T09:36:14.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:17.446 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-06T09:36:18.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:21.474 WARNING:teuthology.provision.fog.smithi018:[Errno None] Unable to connect to port 22 on 172.21.15.18
2023-12-06T09:36:23.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:26.530 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-06T09:36:27.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:30.530 WARNING:teuthology.provision.fog.smithi018:[Errno None] Unable to connect to port 22 on 172.21.15.18
2023-12-06T09:36:32.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:35.590 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-06T09:36:36.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:39.590 WARNING:teuthology.provision.fog.smithi018:[Errno None] Unable to connect to port 22 on 172.21.15.18
2023-12-06T09:36:41.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:44.678 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-06T09:36:45.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:48.678 WARNING:teuthology.provision.fog.smithi018:[Errno None] Unable to connect to port 22 on 172.21.15.18
2023-12-06T09:36:50.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:53.762 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-12-06T09:36:54.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:36:59.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:37:54.735 WARNING:teuthology.provision.fog.smithi018:timed out
2023-12-06T09:37:59.774 WARNING:teuthology.provision.fog.smithi038:timed out
2023-12-06T09:38:00.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:38:00.995 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-06T09:38:00.997 DEBUG:teuthology.orchestra.run.smithi018:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-06T09:38:01.346 INFO:teuthology.provision.fog.smithi018:Node is ready
2023-12-06T09:38:01.346 DEBUG:teuthology.orchestra.run.smithi018:> hostname
2023-12-06T09:38:01.362 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018.front.sepia.ceph.com
2023-12-06T09:38:01.362 DEBUG:teuthology.orchestra.run.smithi018:> grep smithi018.front.sepia.ceph.com /etc/hosts
2023-12-06T09:38:01.418 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18 smithi018.front.sepia.ceph.com smithi018
2023-12-06T09:38:01.419 DEBUG:teuthology.orchestra.run.smithi018:> sudo hostname smithi018
2023-12-06T09:38:01.521 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i -e 's/smithi018.front.sepia.ceph.com/smithi018/g' /etc/hosts
2023-12-06T09:38:01.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i -e 's/172.21.15.18/172.21.15.18/g' /etc/hosts
2023-12-06T09:38:01.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i -e 's/smithi018.front.sepia.ceph.com/smithi018/g' /etc/hostname
2023-12-06T09:38:01.697 DEBUG:teuthology.orchestra.run.smithi018:> sudo hostname smithi018
2023-12-06T09:38:01.768 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release
2023-12-06T09:38:01.823 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream"
2023-12-06T09:38:01.823 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="9"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="9"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el9"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 9"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:LOGO="fedora-logo-icon"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-12-06T09:38:01.824 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-06T09:38:01.825 INFO:teuthology.provision.fog.smithi018:Deploy complete!
2023-12-06T09:38:01.827 DEBUG:teuthology.parallel:result is None
2023-12-06T09:38:05.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:38:06.031 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-06T09:38:06.032 DEBUG:teuthology.orchestra.run.smithi038:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-06T09:38:06.364 INFO:teuthology.provision.fog.smithi038:Node is ready
2023-12-06T09:38:06.365 DEBUG:teuthology.orchestra.run.smithi038:> hostname
2023-12-06T09:38:06.380 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2023-12-06T09:38:06.380 DEBUG:teuthology.orchestra.run.smithi038:> grep smithi038.front.sepia.ceph.com /etc/hosts
2023-12-06T09:38:06.437 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2023-12-06T09:38:06.438 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038
2023-12-06T09:38:06.548 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hosts
2023-12-06T09:38:06.579 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/172.21.15.38/172.21.15.38/g' /etc/hosts
2023-12-06T09:38:06.650 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hostname
2023-12-06T09:38:06.722 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038
2023-12-06T09:38:06.793 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release
2023-12-06T09:38:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream"
2023-12-06T09:38:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9"
2023-12-06T09:38:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos"
2023-12-06T09:38:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-12-06T09:38:06.850 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-06T09:38:06.853 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2023-12-06T09:38:06.854 DEBUG:teuthology.parallel:result is None
2023-12-06T09:38:07.062 INFO:teuthology.lock.ops:Checking smithi018.front.sepia.ceph.com
2023-12-06T09:38:07.062 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-06T09:38:07.087 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com
2023-12-06T09:38:07.087 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-06T09:38:07.104 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: set os type and version on server
2023-12-06T09:38:07.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com'
2023-12-06T09:38:07.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:38:07.203 DEBUG:teuthology.orchestra.run.smithi018:> true
2023-12-06T09:38:07.270 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com'
2023-12-06T09:38:07.270 DEBUG:teuthology.orchestra.run.smithi018:> uname -m
2023-12-06T09:38:07.327 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64
2023-12-06T09:38:07.327 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release
2023-12-06T09:38:07.383 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream"
2023-12-06T09:38:07.383 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="9"
2023-12-06T09:38:07.383 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="9"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el9"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 9"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:LOGO="fedora-logo-icon"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-12-06T09:38:07.384 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-06T09:38:07.385 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server
2023-12-06T09:38:07.404 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server
2023-12-06T09:38:07.404 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2023-12-06T09:38:07.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-06T09:38:07.498 DEBUG:teuthology.orchestra.run.smithi038:> true
2023-12-06T09:38:07.565 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com'
2023-12-06T09:38:07.565 DEBUG:teuthology.orchestra.run.smithi038:> uname -m
2023-12-06T09:38:07.621 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64
2023-12-06T09:38:07.621 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release
2023-12-06T09:38:07.678 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream"
2023-12-06T09:38:07.678 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9"
2023-12-06T09:38:07.678 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos"
2023-12-06T09:38:07.678 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-12-06T09:38:07.679 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-06T09:38:07.680 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2023-12-06T09:38:07.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:38:07.790 INFO:teuthology.dispatcher.supervisor:Running job 7479814
2023-12-06T09:38:07.791 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_alimasa@teuthology --archive /home/teuthworker/archive/alimasa-2023-12-06_09:28:31-rgw-wip-alimasa-multi-pubsub-etag-distro-default-smithi/7479814 --name alimasa-2023-12-06_09:28:31-rgw-wip-alimasa-multi-pubsub-etag-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{centos_latest} tasks/test_multi} -- /home/teuthworker/archive/alimasa-2023-12-06_09:28:31-rgw-wip-alimasa-multi-pubsub-etag-distro-default-smithi/7479814/orig.config.yaml
2023-12-06T09:38:07.794 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/alimasa-2023-12-06_09:28:31-rgw-wip-alimasa-multi-pubsub-etag-distro-default-smithi/7479814
2023-12-06T09:38:07.795 INFO:teuthology.dispatcher.supervisor:Job PID: 1614422
2023-12-06T09:38:07.795 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-06T09:40:07.895 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:42:08.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:44:08.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:46:08.436 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:48:08.600 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:50:08.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:52:08.942 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:54:09.105 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:56:09.377 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T09:58:09.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:00:09.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:02:09.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:04:10.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:06:10.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:08:10.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:10:10.583 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:12:10.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:14:10.955 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:16:11.128 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:18:11.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:20:11.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:22:11.630 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:24:11.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:26:11.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:28:12.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:30:12.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:32:12.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:34:12.675 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:36:12.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:38:13.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:40:13.188 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:42:13.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:44:13.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:46:13.686 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:48:13.857 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:50:14.029 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:52:14.202 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:54:14.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:56:14.532 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T10:58:14.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:00:14.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:02:15.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:04:15.184 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:06:15.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:08:15.518 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:10:15.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:12:15.854 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:14:15.956 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:16:16.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:18:16.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:20:16.448 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:22:16.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:24:16.779 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:26:16.940 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:28:17.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:30:17.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:32:17.415 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:34:17.618 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:36:17.779 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:38:17.941 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:40:18.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:42:18.173 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:44:18.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:46:18.416 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:48:18.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:50:18.724 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:52:18.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:54:19.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:56:19.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T11:58:19.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:00:19.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:02:19.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:04:19.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:06:19.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:08:20.098 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:10:20.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:12:20.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:14:20.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:16:20.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:18:20.960 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:20:21.152 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:22:21.331 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:24:21.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:26:21.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:28:21.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:30:22.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:32:22.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-06T12:32:22.270 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-12-06T12:32:22.450 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-06T12:32:22.450 INFO:teuthology.nuke:Checking targets against current locks
2023-12-06T12:32:22.503 DEBUG:teuthology.nuke:shortname: smithi018
2023-12-06T12:32:22.504 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-06T12:32:22.530 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/alimasa-2023-12-06_09:28:31-rgw-wip-alimasa-multi-pubsub-etag-distro-default-smithi/7479814', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 09:33:15.871773', 'locked_by': 'scheduled_alimasa@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIeVxitXwiZLhVYIp1a7QEhVSth+LVVxA6L/UGntlqSVTndiSoRamBPXPEx6/SM7bWIbnl6i6pGqO1SUAxjXFs='}
2023-12-06T12:32:22.534 DEBUG:teuthology.nuke:shortname: smithi038
2023-12-06T12:32:22.535 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-06T12:32:22.556 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/alimasa-2023-12-06_09:28:31-rgw-wip-alimasa-multi-pubsub-etag-distro-default-smithi/7479814', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 09:33:15.873271', 'locked_by': 'scheduled_alimasa@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF9MhIRIrGfvlXhJRar6gmOc2+Aeqx/O33ZHaJSuzM5wWhEdeAi7XgL3Rwstshw25AbxjSmZ++g/Eu4hef2uYhs='}
2023-12-06T12:32:22.593 INFO:teuthology.orchestra.console.smithi018:Power off
2023-12-06T12:32:22.594 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-06T12:32:22.615 DEBUG:teuthology.orchestra.console.smithi018:power off output: Chassis Power Control: Down/Off
2023-12-06T12:32:22.615 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T12:32:22.632 INFO:teuthology.orchestra.console.smithi038:Power off
2023-12-06T12:32:22.632 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-06T12:32:22.642 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is off
2023-12-06T12:32:22.742 INFO:teuthology.orchestra.console.smithi018:Power off completed
2023-12-06T12:32:22.904 INFO:teuthology.lock.ops:unlocked: smithi018.front.sepia.ceph.com
2023-12-06T12:32:42.701 DEBUG:teuthology.orchestra.console.smithi038:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-12-06T12:32:42.701 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-06T12:33:02.777 DEBUG:teuthology.orchestra.console.smithi038:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-12-06T12:33:02.777 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-06T12:33:15.854 DEBUG:teuthology.orchestra.console.smithi038:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-12-06T12:33:15.855 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-06T12:33:15.966 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off
2023-12-06T12:33:15.966 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-06T12:33:16.003 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off
2023-12-06T12:33:16.104 INFO:teuthology.orchestra.console.smithi038:Power off completed
2023-12-06T12:33:16.253 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com