2024-03-24T19:50:42.060 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-24T19:50:42.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T19:50:42.210 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f8ed048df70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f8ed14eec10>, signals=[15])
2024-03-24T19:50:42.211 INFO:teuthology.lock.ops:Start node 'smithi022.front.sepia.ceph.com' reimaging
2024-03-24T19:50:42.211 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: reset os type and version on server
2024-03-24T19:50:42.211 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2024-03-24T19:50:42.235 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging
2024-03-24T19:50:42.235 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server
2024-03-24T19:50:42.235 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2024-03-24T19:50:42.361 DEBUG:teuthology.provision.fog.smithi104:Requesting image smithi_centos_9.stream (ID 10)
2024-03-24T19:50:42.363 DEBUG:teuthology.provision.fog.smithi022:Requesting image smithi_centos_9.stream (ID 10)
2024-03-24T19:50:42.404 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of centos 9.stream
2024-03-24T19:50:42.407 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of centos 9.stream
2024-03-24T19:50:42.638 INFO:teuthology.orchestra.console.smithi104:Power off
2024-03-24T19:50:42.638 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-24T19:50:42.651 INFO:teuthology.orchestra.console.smithi022:Power off
2024-03-24T19:50:42.651 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-24T19:50:42.661 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off
2024-03-24T19:50:42.661 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:42.672 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off
2024-03-24T19:50:42.673 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:42.683 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on
2024-03-24T19:50:42.694 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on
2024-03-24T19:50:46.686 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:46.695 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:46.797 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on
2024-03-24T19:50:46.804 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on
2024-03-24T19:50:50.798 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:50.805 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:50.907 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off
2024-03-24T19:50:50.913 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-03-24T19:50:51.009 INFO:teuthology.orchestra.console.smithi104:Power off completed
2024-03-24T19:50:51.014 INFO:teuthology.orchestra.console.smithi022:Power off completed
2024-03-24T19:50:51.109 INFO:teuthology.orchestra.console.smithi104:Power on
2024-03-24T19:50:51.109 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-24T19:50:51.118 INFO:teuthology.orchestra.console.smithi022:Power on
2024-03-24T19:50:51.118 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-24T19:50:51.131 DEBUG:teuthology.orchestra.console.smithi104:power on output: Chassis Power Control: Up/On
2024-03-24T19:50:51.132 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:51.141 DEBUG:teuthology.orchestra.console.smithi022:power on output: Chassis Power Control: Up/On
2024-03-24T19:50:51.141 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:51.154 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off
2024-03-24T19:50:51.161 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-03-24T19:50:55.158 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:55.167 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:55.269 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off
2024-03-24T19:50:55.275 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off
2024-03-24T19:50:59.274 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:59.282 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-24T19:50:59.385 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on
2024-03-24T19:50:59.486 INFO:teuthology.orchestra.console.smithi104:Power on completed
2024-03-24T19:50:59.487 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on
2024-03-24T19:50:59.587 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish
2024-03-24T19:50:59.591 INFO:teuthology.orchestra.console.smithi022:Power on completed
2024-03-24T19:50:59.691 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish
2024-03-24T19:53:30.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:53:30.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:53:33.630 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-03-24T19:53:33.631 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2024-03-24T19:53:39.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:53:39.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:53:42.686 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-03-24T19:53:42.687 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2024-03-24T19:53:48.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:53:48.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:53:51.746 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-03-24T19:53:51.747 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2024-03-24T19:53:57.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:53:57.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:54:00.830 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-03-24T19:54:00.831 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2024-03-24T19:54:06.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:54:06.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:54:09.890 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22
2024-03-24T19:54:09.922 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2024-03-24T19:54:15.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:54:15.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:54:18.974 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2024-03-24T19:54:24.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:55:15.947 WARNING:teuthology.provision.fog.smithi022:timed out
2024-03-24T19:55:21.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:55:22.268 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-24T19:55:22.269 DEBUG:teuthology.orchestra.run.smithi022:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-24T19:55:22.620 INFO:teuthology.provision.fog.smithi022:Node is ready
2024-03-24T19:55:22.620 DEBUG:teuthology.orchestra.run.smithi022:> hostname
2024-03-24T19:55:22.636 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com
2024-03-24T19:55:22.636 DEBUG:teuthology.orchestra.run.smithi022:> grep smithi022.front.sepia.ceph.com /etc/hosts
2024-03-24T19:55:22.691 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2024-03-24T19:55:22.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022
2024-03-24T19:55:22.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hosts
2024-03-24T19:55:22.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/172.21.15.22/172.21.15.22/g' /etc/hosts
2024-03-24T19:55:22.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hostname
2024-03-24T19:55:22.975 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022
2024-03-24T19:55:23.046 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release
2024-03-24T19:55:23.101 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream"
2024-03-24T19:55:23.101 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-24T19:55:23.102 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-24T19:55:23.103 INFO:teuthology.provision.fog.smithi022:Deploy complete!
2024-03-24T19:55:23.104 DEBUG:teuthology.parallel:result is None
2024-03-24T19:55:24.982 WARNING:teuthology.provision.fog.smithi104:timed out
2024-03-24T19:55:30.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:55:31.214 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-24T19:55:31.215 DEBUG:teuthology.orchestra.run.smithi104:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-24T19:55:31.545 INFO:teuthology.provision.fog.smithi104:Node is ready
2024-03-24T19:55:31.546 DEBUG:teuthology.orchestra.run.smithi104:> hostname
2024-03-24T19:55:31.560 INFO:teuthology.orchestra.run.smithi104.stdout:smithi104.front.sepia.ceph.com
2024-03-24T19:55:31.560 DEBUG:teuthology.orchestra.run.smithi104:> grep smithi104.front.sepia.ceph.com /etc/hosts
2024-03-24T19:55:31.615 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.104 smithi104.front.sepia.ceph.com smithi104
2024-03-24T19:55:31.616 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104
2024-03-24T19:55:31.757 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hosts
2024-03-24T19:55:31.785 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/172.21.15.104/172.21.15.104/g' /etc/hosts
2024-03-24T19:55:31.853 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hostname
2024-03-24T19:55:31.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104
2024-03-24T19:55:31.988 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon"
2024-03-24T19:55:32.042 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-24T19:55:32.043 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/"
2024-03-24T19:55:32.043 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-24T19:55:32.043 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-24T19:55:32.043 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-24T19:55:32.043 INFO:teuthology.provision.fog.smithi104:Deploy complete!
2024-03-24T19:55:32.044 DEBUG:teuthology.parallel:result is None
2024-03-24T19:55:32.217 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com
2024-03-24T19:55:32.217 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-24T19:55:32.245 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com
2024-03-24T19:55:32.245 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-24T19:55:32.271 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server
2024-03-24T19:55:32.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com'
2024-03-24T19:55:32.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:55:32.378 DEBUG:teuthology.orchestra.run.smithi022:> true
2024-03-24T19:55:32.451 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com'
2024-03-24T19:55:32.451 DEBUG:teuthology.orchestra.run.smithi022:> uname -m
2024-03-24T19:55:32.508 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64
2024-03-24T19:55:32.509 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release
2024-03-24T19:55:32.565 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream"
2024-03-24T19:55:32.565 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9"
2024-03-24T19:55:32.565 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos"
2024-03-24T19:55:32.565 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora"
2024-03-24T19:55:32.565 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-24T19:55:32.566 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-24T19:55:32.567 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2024-03-24T19:55:32.595 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server
2024-03-24T19:55:32.595 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com'
2024-03-24T19:55:32.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-24T19:55:32.681 DEBUG:teuthology.orchestra.run.smithi104:> true
2024-03-24T19:55:32.749 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com'
2024-03-24T19:55:32.749 DEBUG:teuthology.orchestra.run.smithi104:> uname -m
2024-03-24T19:55:32.804 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64
2024-03-24T19:55:32.804 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-24T19:55:32.859 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/"
2024-03-24T19:55:32.860 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-24T19:55:32.860 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-24T19:55:32.860 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-24T19:55:32.860 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2024-03-24T19:55:32.878 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T19:55:32.974 INFO:teuthology.dispatcher.supervisor:Running job 7619307
2024-03-24T19:55:32.974 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_amaredia@teuthology --archive /home/teuthworker/archive/amaredia-2024-03-24_19:47:18-rgw:multisite-wip-rgw-multisite-sync-delta-counters-distro-default-smithi/7619307 --name amaredia-2024-03-24_19:47:18-rgw:multisite-wip-rgw-multisite-sync-delta-counters-distro-default-smithi --description rgw:multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{centos_latest} tasks/test_multi} -- /home/teuthworker/archive/amaredia-2024-03-24_19:47:18-rgw:multisite-wip-rgw-multisite-sync-delta-counters-distro-default-smithi/7619307/orig.config.yaml
2024-03-24T19:55:32.979 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/amaredia-2024-03-24_19:47:18-rgw:multisite-wip-rgw-multisite-sync-delta-counters-distro-default-smithi/7619307
2024-03-24T19:55:32.979 INFO:teuthology.dispatcher.supervisor:Job PID: 2600990
2024-03-24T19:55:32.979 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-24T19:57:33.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T19:59:33.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:01:33.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:03:33.599 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:05:33.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:07:33.947 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:09:34.115 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:11:34.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:13:34.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:15:34.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:17:34.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:19:34.970 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:21:35.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:23:35.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:25:35.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:27:35.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:29:35.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:31:35.955 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:33:36.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:35:36.311 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:37:36.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:39:36.650 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:41:36.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:43:36.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:45:37.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:47:37.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:49:37.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:51:37.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:53:37.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:55:37.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:57:38.169 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T20:59:38.331 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:01:38.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:03:38.654 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:05:38.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:07:38.995 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:09:39.175 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:11:39.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:13:39.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:15:39.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:17:39.895 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:19:40.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:21:40.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:23:40.433 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:25:40.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:27:40.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:29:41.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:31:41.170 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:33:41.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:35:41.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:37:41.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:39:41.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:41:41.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:43:42.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:45:42.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:47:42.428 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:49:42.599 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:51:42.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:53:42.949 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:55:43.103 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:57:43.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T21:59:43.436 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:01:43.599 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:03:43.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:05:43.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:07:44.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:09:44.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:11:44.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:13:44.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:15:44.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:17:44.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:19:45.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:21:45.320 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:23:45.480 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:25:45.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:27:45.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:29:45.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:31:46.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:33:46.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:35:46.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:37:46.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:39:46.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:41:46.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:43:47.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:45:47.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:47:47.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:49:47.675 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:51:47.846 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:53:48.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:55:48.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:57:48.572 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T22:59:48.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T23:01:48.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T23:03:49.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-24T23:03:49.163 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1