2024-03-26T17:58:00.718 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-26T17:58:00.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T17:58:00.909 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fe814a7df40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fe815adec10>, signals=[15])
2024-03-26T17:58:00.909 INFO:teuthology.lock.ops:Start node 'smithi110.front.sepia.ceph.com' reimaging
2024-03-26T17:58:00.909 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: reset os type and version on server
2024-03-26T17:58:00.910 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2024-03-26T17:58:00.933 INFO:teuthology.lock.ops:Start node 'smithi133.front.sepia.ceph.com' reimaging
2024-03-26T17:58:00.933 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: reset os type and version on server
2024-03-26T17:58:00.933 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server
2024-03-26T17:58:00.957 INFO:teuthology.lock.ops:Start node 'smithi193.front.sepia.ceph.com' reimaging
2024-03-26T17:58:00.957 INFO:teuthology.lock.ops:Updating [smithi193.front.sepia.ceph.com]: reset os type and version on server
2024-03-26T17:58:00.957 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server
2024-03-26T17:58:01.039 DEBUG:teuthology.provision.fog.smithi110:Requesting image smithi_centos_9.stream (ID 10)
2024-03-26T17:58:01.068 DEBUG:teuthology.provision.fog.smithi133:Requesting image smithi_centos_9.stream (ID 10)
2024-03-26T17:58:01.075 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 9.stream
2024-03-26T17:58:01.087 DEBUG:teuthology.provision.fog.smithi193:Requesting image smithi_centos_9.stream (ID 10)
2024-03-26T17:58:01.113 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of centos 9.stream
2024-03-26T17:58:01.131 INFO:teuthology.provision.fog.smithi193:Scheduling deploy of centos 9.stream
2024-03-26T17:58:01.213 DEBUG:teuthology.provision.fog.smithi193:Canceling deploy task with ID 897568
2024-03-26T17:58:01.402 INFO:teuthology.orchestra.console.smithi110:Power off
2024-03-26T17:58:01.403 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-26T17:58:01.416 INFO:teuthology.orchestra.console.smithi133:Power off
2024-03-26T17:58:01.417 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-26T17:58:01.424 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off
2024-03-26T17:58:01.425 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:01.437 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off
2024-03-26T17:58:01.437 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:01.450 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2024-03-26T17:58:01.460 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off
2024-03-26T17:58:01.471 INFO:teuthology.orchestra.console.smithi193:Power off
2024-03-26T17:58:01.471 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-26T17:58:01.494 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off
2024-03-26T17:58:01.494 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:01.515 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off
2024-03-26T17:58:01.552 INFO:teuthology.orchestra.console.smithi110:Power off completed
2024-03-26T17:58:01.561 INFO:teuthology.orchestra.console.smithi133:Power off completed
2024-03-26T17:58:01.616 INFO:teuthology.orchestra.console.smithi193:Power off completed
2024-03-26T17:58:01.653 INFO:teuthology.orchestra.console.smithi110:Power on
2024-03-26T17:58:01.653 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-26T17:58:01.662 INFO:teuthology.orchestra.console.smithi133:Power on
2024-03-26T17:58:01.662 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-26T17:58:01.676 DEBUG:teuthology.orchestra.console.smithi110:power on output: Chassis Power Control: Up/On
2024-03-26T17:58:01.676 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:01.685 DEBUG:teuthology.orchestra.console.smithi133:power on output: Chassis Power Control: Up/On
2024-03-26T17:58:01.685 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:01.708 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off
2024-03-26T17:58:01.711 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2024-03-26T17:58:01.716 INFO:teuthology.orchestra.console.smithi193:Power on
2024-03-26T17:58:01.716 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-26T17:58:01.737 DEBUG:teuthology.orchestra.console.smithi193:power on output: Chassis Power Control: Up/On
2024-03-26T17:58:01.738 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:01.760 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off
2024-03-26T17:58:05.710 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:05.718 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:05.761 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:05.821 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off
2024-03-26T17:58:05.827 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2024-03-26T17:58:05.870 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off
2024-03-26T17:58:05.928 INFO:teuthology.orchestra.console.smithi110:Power on completed
2024-03-26T17:58:06.029 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish
2024-03-26T17:58:09.826 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:09.871 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-26T17:58:09.936 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on
2024-03-26T17:58:09.979 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on
2024-03-26T17:58:10.037 INFO:teuthology.orchestra.console.smithi133:Power on completed
2024-03-26T17:58:10.080 INFO:teuthology.orchestra.console.smithi193:Power on completed
2024-03-26T17:58:10.138 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish
2024-03-26T17:58:10.181 INFO:teuthology.provision.fog.smithi193:Waiting for deploy to finish
2024-03-26T17:58:10.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:58:12.286 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:58:18.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:58:21.378 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:58:27.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:58:30.466 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:58:36.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:58:39.522 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:58:45.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:58:48.610 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:58:54.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:58:57.662 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:59:03.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:59:06.754 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:59:12.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:59:31.106 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:59:37.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:59:40.083 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T17:59:46.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T17:59:46.224 ERROR:teuthology.orchestra.connection:Error authenticating with smithi193.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-26T17:59:50.314 ERROR:teuthology.orchestra.connection:Error authenticating with smithi193.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-26T17:59:57.419 ERROR:teuthology.orchestra.connection:Error authenticating with smithi193.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-26T18:00:07.511 ERROR:teuthology.orchestra.connection:Error authenticating with smithi193.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-26T18:00:20.642 ERROR:teuthology.orchestra.connection:Error authenticating with smithi193.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-26T18:00:37.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:00:40.191 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2024-03-26T18:00:46.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:00:49.250 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2024-03-26T18:00:55.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:00:56.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:00:58.303 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2024-03-26T18:00:59.294 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-26T18:01:04.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:05.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:07.358 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2024-03-26T18:01:08.350 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-26T18:01:13.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:14.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:16.414 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2024-03-26T18:01:17.406 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-26T18:01:22.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:23.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:25.470 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2024-03-26T18:01:26.462 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-26T18:01:31.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:32.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:35.518 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-26T18:01:36.645 WARNING:teuthology.provision.fog.smithi193:timed out
2024-03-26T18:01:41.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:42.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:44.574 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-26T18:01:45.726 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T18:01:50.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:51.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:01:54.782 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T18:02:00.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:02:03.842 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T18:02:09.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:02:12.894 WARNING:teuthology.provision.fog.smithi193:[Errno None] Unable to connect to port 22 on 172.21.15.193
2024-03-26T18:02:18.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:02:19.210 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-26T18:02:19.211 DEBUG:teuthology.orchestra.run.smithi193:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-26T18:02:31.474 WARNING:teuthology.provision.fog.smithi110:timed out
2024-03-26T18:02:34.682 INFO:teuthology.provision.fog.smithi193:Node is ready
2024-03-26T18:02:34.682 DEBUG:teuthology.orchestra.run.smithi193:> hostname
2024-03-26T18:02:34.702 INFO:teuthology.orchestra.run.smithi193.stdout:smithi193.front.sepia.ceph.com
2024-03-26T18:02:34.702 DEBUG:teuthology.orchestra.run.smithi193:> grep smithi193.front.sepia.ceph.com /etc/hosts
2024-03-26T18:02:34.759 INFO:teuthology.orchestra.run.smithi193.stdout:172.21.15.193 smithi193.front.sepia.ceph.com smithi193
2024-03-26T18:02:34.759 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname smithi193
2024-03-26T18:02:34.907 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i -e 's/smithi193.front.sepia.ceph.com/smithi193/g' /etc/hosts
2024-03-26T18:02:34.936 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i -e 's/172.21.15.193/172.21.15.193/g' /etc/hosts
2024-03-26T18:02:35.005 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i -e 's/smithi193.front.sepia.ceph.com/smithi193/g' /etc/hostname
2024-03-26T18:02:35.074 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname smithi193
2024-03-26T18:02:35.142 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-26T18:02:35.197 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/"
2024-03-26T18:02:35.198 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-26T18:02:35.198 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-26T18:02:35.198 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-26T18:02:35.198 INFO:teuthology.provision.fog.smithi193:Deploy complete!
2024-03-26T18:02:35.199 DEBUG:teuthology.parallel:result is None
2024-03-26T18:02:37.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:02:37.746 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-26T18:02:37.746 DEBUG:teuthology.orchestra.run.smithi110:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-26T18:02:38.077 INFO:teuthology.provision.fog.smithi110:Node is ready
2024-03-26T18:02:38.077 DEBUG:teuthology.orchestra.run.smithi110:> hostname
2024-03-26T18:02:38.091 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com
2024-03-26T18:02:38.091 DEBUG:teuthology.orchestra.run.smithi110:> grep smithi110.front.sepia.ceph.com /etc/hosts
2024-03-26T18:02:38.146 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110
2024-03-26T18:02:38.146 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110
2024-03-26T18:02:38.258 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hosts
2024-03-26T18:02:38.286 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/172.21.15.110/172.21.15.110/g' /etc/hosts
2024-03-26T18:02:38.353 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hostname
2024-03-26T18:02:38.420 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110
2024-03-26T18:02:38.486 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release
2024-03-26T18:02:38.540 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream"
2024-03-26T18:02:38.540 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-26T18:02:38.541 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-26T18:02:38.542 INFO:teuthology.provision.fog.smithi110:Deploy complete!
2024-03-26T18:02:38.542 DEBUG:teuthology.parallel:result is None
2024-03-26T18:02:50.578 WARNING:teuthology.provision.fog.smithi133:timed out
2024-03-26T18:02:56.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:02:56.882 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-26T18:02:56.882 DEBUG:teuthology.orchestra.run.smithi133:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-26T18:02:57.202 INFO:teuthology.provision.fog.smithi133:Node is ready
2024-03-26T18:02:57.203 DEBUG:teuthology.orchestra.run.smithi133:> hostname
2024-03-26T18:02:57.217 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com
2024-03-26T18:02:57.217 DEBUG:teuthology.orchestra.run.smithi133:> grep smithi133.front.sepia.ceph.com /etc/hosts
2024-03-26T18:02:57.272 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133
2024-03-26T18:02:57.273 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname smithi133
2024-03-26T18:02:57.371 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/smithi133.front.sepia.ceph.com/smithi133/g' /etc/hosts
2024-03-26T18:02:57.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/172.21.15.133/172.21.15.133/g' /etc/hosts
2024-03-26T18:02:57.466 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/smithi133.front.sepia.ceph.com/smithi133/g' /etc/hostname
2024-03-26T18:02:57.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname smithi133
2024-03-26T18:02:57.599 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release
2024-03-26T18:02:57.653 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream"
2024-03-26T18:02:57.653 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9"
2024-03-26T18:02:57.653 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos"
2024-03-26T18:02:57.653 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-26T18:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-26T18:02:57.654 INFO:teuthology.provision.fog.smithi133:Deploy complete!
2024-03-26T18:02:57.655 DEBUG:teuthology.parallel:result is None
2024-03-26T18:02:57.867 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com
2024-03-26T18:02:57.868 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-26T18:02:58.202 INFO:teuthology.lock.ops:Checking smithi133.front.sepia.ceph.com
2024-03-26T18:02:58.202 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-26T18:02:58.226 INFO:teuthology.lock.ops:Checking smithi193.front.sepia.ceph.com
2024-03-26T18:02:58.228 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server
2024-03-26T18:02:58.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com'
2024-03-26T18:02:58.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:02:58.319 DEBUG:teuthology.orchestra.run.smithi110:> true
2024-03-26T18:02:58.493 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com'
2024-03-26T18:02:58.493 DEBUG:teuthology.orchestra.run.smithi110:> uname -m
2024-03-26T18:02:58.547 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64
2024-03-26T18:02:58.548 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream"
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9"
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos"
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora"
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9"
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9"
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2024-03-26T18:02:58.602 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon"
2024-03-26T18:02:58.603 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-26T18:02:58.603 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/"
2024-03-26T18:02:58.603 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-26T18:02:58.603 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-26T18:02:58.603 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-26T18:02:58.603 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2024-03-26T18:02:58.627 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: set os type and version on server
2024-03-26T18:02:58.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com'
2024-03-26T18:02:58.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:02:58.713 DEBUG:teuthology.orchestra.run.smithi133:> true
2024-03-26T18:02:58.780 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com'
2024-03-26T18:02:58.780 DEBUG:teuthology.orchestra.run.smithi133:> uname -m
2024-03-26T18:02:58.835 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64
2024-03-26T18:02:58.835 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release
2024-03-26T18:02:58.890 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream"
2024-03-26T18:02:58.890 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9"
2024-03-26T18:02:58.890 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos"
2024-03-26T18:02:58.890 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora"
2024-03-26T18:02:58.890 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9"
2024-03-26T18:02:58.890 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9"
2024-03-26T18:02:58.890 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-26T18:02:58.891 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31"
2024-03-26T18:02:58.891 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon"
2024-03-26T18:02:58.891 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-26T18:02:58.891 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/"
2024-03-26T18:02:58.891 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-26T18:02:58.891 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-26T18:02:58.891 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-26T18:02:58.891 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server
2024-03-26T18:02:58.910 INFO:teuthology.lock.ops:Updating [smithi193.front.sepia.ceph.com]: set os type and version on server
2024-03-26T18:02:58.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com'
2024-03-26T18:02:58.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-26T18:02:59.003 DEBUG:teuthology.orchestra.run.smithi193:> true
2024-03-26T18:02:59.068 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com'
2024-03-26T18:02:59.068 DEBUG:teuthology.orchestra.run.smithi193:> uname -m
2024-03-26T18:02:59.123 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64
2024-03-26T18:02:59.124 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream"
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9"
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos"
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora"
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9"
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9"
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31"
2024-03-26T18:02:59.178 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon"
2024-03-26T18:02:59.179 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-26T18:02:59.179 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/"
2024-03-26T18:02:59.179 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-26T18:02:59.179 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-26T18:02:59.179 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-26T18:02:59.179 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server
2024-03-26T18:02:59.202 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T18:02:59.334 INFO:teuthology.dispatcher.supervisor:Running job 7623788
2024-03-26T18:02:59.334 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623788 --name lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} -- /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623788/orig.config.yaml
2024-03-26T18:02:59.337 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623788
2024-03-26T18:02:59.338 INFO:teuthology.dispatcher.supervisor:Job PID: 3003720
2024-03-26T18:02:59.338 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-26T18:04:59.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T18:06:59.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T18:08:59.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T18:10:59.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T18:12:59.618 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T18:14:59.689 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T18:16:59.749 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-26T18:16:59.814 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-03-26T18:16:59.964 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi193 but it was locked by another job: /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623789