2024-03-07T13:29:11.012 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-07T13:29:11.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:29:11.146 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f8509de8f70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f850ae49c10>, signals=[15])
2024-03-07T13:29:11.147 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging
2024-03-07T13:29:11.147 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server
2024-03-07T13:29:11.147 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2024-03-07T13:29:11.167 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging
2024-03-07T13:29:11.167 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server
2024-03-07T13:29:11.168 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2024-03-07T13:29:11.298 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_centos_9.stream (ID 10)
2024-03-07T13:29:11.309 DEBUG:teuthology.provision.fog.smithi191:Requesting image smithi_centos_9.stream (ID 10)
2024-03-07T13:29:11.347 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 9.stream
2024-03-07T13:29:11.359 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of centos 9.stream
2024-03-07T13:29:11.563 INFO:teuthology.orchestra.console.smithi159:Power off
2024-03-07T13:29:11.563 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T13:29:11.579 INFO:teuthology.orchestra.console.smithi191:Power off
2024-03-07T13:29:11.579 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T13:29:11.588 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off
2024-03-07T13:29:11.588 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:11.602 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off
2024-03-07T13:29:11.603 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:11.611 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2024-03-07T13:29:11.628 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on
2024-03-07T13:29:15.616 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:15.629 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:15.726 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2024-03-07T13:29:15.739 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on
2024-03-07T13:29:19.728 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:19.740 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:19.838 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2024-03-07T13:29:19.850 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on
2024-03-07T13:29:19.939 INFO:teuthology.orchestra.console.smithi159:Power off completed
2024-03-07T13:29:20.040 INFO:teuthology.orchestra.console.smithi159:Power on
2024-03-07T13:29:20.040 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-07T13:29:20.064 DEBUG:teuthology.orchestra.console.smithi159:power on output: Chassis Power Control: Up/On
2024-03-07T13:29:20.064 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:20.089 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2024-03-07T13:29:23.854 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:23.963 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off
2024-03-07T13:29:24.064 INFO:teuthology.orchestra.console.smithi191:Power off completed
2024-03-07T13:29:24.090 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:24.165 INFO:teuthology.orchestra.console.smithi191:Power on
2024-03-07T13:29:24.166 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-07T13:29:24.190 DEBUG:teuthology.orchestra.console.smithi191:power on output: Chassis Power Control: Up/On
2024-03-07T13:29:24.190 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:24.201 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2024-03-07T13:29:24.213 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off
2024-03-07T13:29:28.206 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:28.218 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T13:29:28.320 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2024-03-07T13:29:28.331 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on
2024-03-07T13:29:28.422 INFO:teuthology.orchestra.console.smithi159:Power on completed
2024-03-07T13:29:28.432 INFO:teuthology.orchestra.console.smithi191:Power on completed
2024-03-07T13:29:28.523 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish
2024-03-07T13:29:28.533 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish
2024-03-07T13:32:14.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:14.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:17.440 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191
2024-03-07T13:32:17.440 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:32:23.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:23.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:26.524 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191
2024-03-07T13:32:26.524 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:32:32.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:32.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:35.616 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191
2024-03-07T13:32:35.616 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:32:41.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:41.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:44.700 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:32:44.700 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191
2024-03-07T13:32:50.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:50.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:53.788 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:32:53.788 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191
2024-03-07T13:32:59.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:32:59.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:33:02.876 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:33:02.877 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191
2024-03-07T13:33:08.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:33:08.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:33:11.933 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:33:17.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:33:36.444 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:33:42.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:33:45.504 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:33:51.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:33:54.588 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-03-07T13:34:00.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:34:01.133 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-07T13:34:01.133 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-07T13:34:08.885 WARNING:teuthology.provision.fog.smithi191:timed out
2024-03-07T13:34:14.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:34:15.197 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-07T13:34:15.197 DEBUG:teuthology.orchestra.run.smithi191:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-07T13:34:15.521 INFO:teuthology.provision.fog.smithi191:Node is ready
2024-03-07T13:34:15.521 DEBUG:teuthology.orchestra.run.smithi191:> hostname
2024-03-07T13:34:15.537 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com
2024-03-07T13:34:15.537 DEBUG:teuthology.orchestra.run.smithi191:> grep smithi191.front.sepia.ceph.com /etc/hosts
2024-03-07T13:34:15.592 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191
2024-03-07T13:34:15.593 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191
2024-03-07T13:34:15.698 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hosts
2024-03-07T13:34:15.726 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/172.21.15.191/172.21.15.191/g' /etc/hosts
2024-03-07T13:34:15.794 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hostname
2024-03-07T13:34:15.861 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191
2024-03-07T13:34:15.928 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T13:34:15.982 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/"
2024-03-07T13:34:15.983 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T13:34:15.983 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T13:34:15.983 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T13:34:15.983 INFO:teuthology.provision.fog.smithi191:Deploy complete!
2024-03-07T13:34:15.985 DEBUG:teuthology.parallel:result is None
2024-03-07T13:34:16.580 INFO:teuthology.provision.fog.smithi159:Node is ready
2024-03-07T13:34:16.580 DEBUG:teuthology.orchestra.run.smithi159:> hostname
2024-03-07T13:34:16.597 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com
2024-03-07T13:34:16.597 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi159.front.sepia.ceph.com /etc/hosts
2024-03-07T13:34:16.654 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159
2024-03-07T13:34:16.654 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159
2024-03-07T13:34:16.760 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hosts
2024-03-07T13:34:16.789 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.159/172.21.15.159/g' /etc/hosts
2024-03-07T13:34:16.857 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hostname
2024-03-07T13:34:16.924 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159
2024-03-07T13:34:16.991 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release
2024-03-07T13:34:17.046 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream"
2024-03-07T13:34:17.046 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T13:34:17.047 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T13:34:17.047 INFO:teuthology.provision.fog.smithi159:Deploy complete!
2024-03-07T13:34:17.048 DEBUG:teuthology.parallel:result is None
2024-03-07T13:34:17.216 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com
2024-03-07T13:34:17.216 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-07T13:34:17.238 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com
2024-03-07T13:34:17.239 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-07T13:34:17.256 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server
2024-03-07T13:34:17.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com'
2024-03-07T13:34:17.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:34:17.347 DEBUG:teuthology.orchestra.run.smithi159:> true
2024-03-07T13:34:17.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com'
2024-03-07T13:34:17.415 DEBUG:teuthology.orchestra.run.smithi159:> uname -m
2024-03-07T13:34:17.469 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64
2024-03-07T13:34:17.469 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream"
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9"
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos"
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora"
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9"
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9"
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2024-03-07T13:34:17.524 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon"
2024-03-07T13:34:17.525 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T13:34:17.525 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/"
2024-03-07T13:34:17.525 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T13:34:17.525 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T13:34:17.525 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T13:34:17.525 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2024-03-07T13:34:17.547 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server
2024-03-07T13:34:17.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com'
2024-03-07T13:34:17.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T13:34:17.640 DEBUG:teuthology.orchestra.run.smithi191:> true
2024-03-07T13:34:17.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com'
2024-03-07T13:34:17.709 DEBUG:teuthology.orchestra.run.smithi191:> uname -m
2024-03-07T13:34:17.765 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64
2024-03-07T13:34:17.766 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release
2024-03-07T13:34:17.821 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T13:34:17.822 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T13:34:17.823 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2024-03-07T13:34:17.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:34:17.931 INFO:teuthology.dispatcher.supervisor:Running job 7584476
2024-03-07T13:34:17.932 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/7584476 --name vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} -- /home/teuthworker/archive/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/7584476/orig.config.yaml
2024-03-07T13:34:17.935 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/7584476
2024-03-07T13:34:17.936 INFO:teuthology.dispatcher.supervisor:Job PID: 644982
2024-03-07T13:34:17.936 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-07T13:36:18.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:38:18.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:40:18.365 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:42:18.533 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:44:18.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:46:18.881 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:48:19.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T13:49:00.582 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:50:02.487 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 504: <html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:50:59.433 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:03.150 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:07.751 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:13.251 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:19.646 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:26.944 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:35.150 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:44.246 ERROR:teuthology.report:POST to https://paddles.front.sepia.ceph.com/runs/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/jobs/ failed with status 502: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

2024-03-07T13:51:44.246 ERROR:teuthology.dispatcher.supervisor:Failed to report job status; ignoring
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 335, in run_with_watchdog
    report.try_push_job_info(job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/report.py", line 512, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/report.py", line 474, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/report.py", line 299, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7584476' reached maximum tries (11) after waiting for 50.41090196721279 seconds
2024-03-07T13:53:44.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T19:10:12.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T19:10:12.979 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1