2023-07-21T17:17:14.608 INFO:root:teuthology version: 0.0.1.dev153+g79491cb
2023-07-21T17:17:14.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T17:17:14.611 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T17:17:34.695 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f1ade776550>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f1adf9918b0>, signals=[15])
2023-07-21T17:17:34.696 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging
2023-07-21T17:17:34.697 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server
2023-07-21T17:17:34.697 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server
2023-07-21T17:17:34.730 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging
2023-07-21T17:17:34.732 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server
2023-07-21T17:17:34.732 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2023-07-21T17:17:34.734 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /host {"name": "smithi140"}
2023-07-21T17:17:39.763 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /host {"name": "smithi146"}
2023-07-21T17:17:49.801 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-07-21T17:17:54.757 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-07-21T17:17:54.780 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_centos_9.stream (ID 10)
2023-07-21T17:17:54.780 DEBUG:teuthology.provision.fog.smithi140:FOG request: PUT /host/245 {"imageID": 10}
2023-07-21T17:17:54.808 DEBUG:teuthology.provision.fog.smithi146:Requesting image smithi_centos_9.stream (ID 10)
2023-07-21T17:17:54.809 DEBUG:teuthology.provision.fog.smithi146:FOG request: PUT /host/251 {"imageID": 10}
2023-07-21T17:17:54.834 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 9.stream
2023-07-21T17:17:54.834 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:17:54.866 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 9.stream
2023-07-21T17:17:54.867 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None
2023-07-21T17:17:59.874 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /tasktype {"name": "deploy"}
2023-07-21T17:18:04.930 DEBUG:teuthology.provision.fog.smithi140:FOG request: POST /host/245/task {"taskTypeID": 1}
2023-07-21T17:18:04.977 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /tasktype {"name": "deploy"}
2023-07-21T17:18:05.017 DEBUG:teuthology.provision.fog.smithi146:FOG request: POST /host/251/task {"taskTypeID": 1}
2023-07-21T17:18:05.026 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:18:05.084 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None
2023-07-21T17:18:15.150 INFO:teuthology.orchestra.console:Power off smithi140
2023-07-21T17:18:15.151 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-21T17:18:15.165 INFO:teuthology.orchestra.console:Power off smithi146
2023-07-21T17:18:15.166 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-21T17:18:15.175 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:15.188 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:19.206 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:19.214 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:24.317 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:24.325 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:29.429 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:29.438 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:29.641 INFO:teuthology.orchestra.console:Power off for smithi140 completed
2023-07-21T17:18:29.743 INFO:teuthology.orchestra.console:Power on smithi140
2023-07-21T17:18:29.743 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-21T17:18:29.764 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:34.552 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:34.779 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:39.665 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:39.889 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:44.777 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:45.001 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:49.890 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:50.113 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:55.008 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:18:55.223 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:00.119 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:00.330 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2023-07-21T17:19:00.334 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:00.432 INFO:teuthology.orchestra.console:Power on smithi146
2023-07-21T17:19:00.433 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-21T17:19:00.455 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:00.544 INFO:teuthology.orchestra.console:Power on for smithi140 completed
2023-07-21T17:19:00.645 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish
2023-07-21T17:19:00.645 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:19:04.479 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:09.590 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:14.704 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:18.044 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:19:20.247 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:25.360 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:30.475 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:35.340 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:19:35.588 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:40.699 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:45.812 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:19:46.023 INFO:teuthology.orchestra.console:Power on for smithi146 completed
2023-07-21T17:19:46.124 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2023-07-21T17:19:46.124 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None
2023-07-21T17:19:51.129 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:19:52.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:19:55.440 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-07-21T17:20:01.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:20:07.387 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:20:12.393 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-07-21T17:20:18.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:20:26.484 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-07-21T17:20:27.450 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:20:32.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:20:38.612 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-07-21T17:20:42.516 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:20:44.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:20:47.696 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-07-21T17:20:53.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:20:53.913 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2023-07-21T17:20:57.588 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:21:07.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:21:12.682 ERROR:teuthology.orchestra.connection:Error authenticating with smithi146.front.sepia.ceph.com: Authentication failed.
2023-07-21T17:21:12.683 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-21T17:21:12.684 DEBUG:teuthology.orchestra.run.smithi146:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-21T17:21:12.685 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:13.690 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:14.695 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:15.701 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:16.706 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:17.712 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:18.717 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:19.723 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:20.728 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:21.734 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:22.739 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:23.744 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:24.750 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:25.755 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:26.761 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:27.679 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None
2023-07-21T17:21:32.686 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:32.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:21:33.695 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:34.701 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:35.707 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:35.860 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-07-21T17:21:36.713 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:37.719 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:38.725 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:39.734 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:40.741 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:41.750 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T17:21:41.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:21:49.936 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-07-21T17:21:55.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:21:58.992 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-07-21T17:22:04.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:22:08.048 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-07-21T17:22:14.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:22:17.104 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-07-21T17:22:23.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:22:26.164 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-07-21T17:22:32.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:23:32.168 WARNING:teuthology.provision.fog.smithi140:timed out
2023-07-21T17:23:38.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:23:38.364 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-21T17:23:38.364 DEBUG:teuthology.orchestra.run.smithi140:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-21T17:23:38.684 INFO:teuthology.provision.fog.smithi140:Node is ready
2023-07-21T17:23:38.685 DEBUG:teuthology.orchestra.run.smithi140:> hostname
2023-07-21T17:23:38.700 INFO:teuthology.orchestra.run.smithi140.stdout:smithi140.front.sepia.ceph.com
2023-07-21T17:23:38.701 DEBUG:teuthology.orchestra.run.smithi140:> grep smithi140.front.sepia.ceph.com /etc/hosts
2023-07-21T17:23:38.755 INFO:teuthology.orchestra.run.smithi140.stdout:172.21.15.140 smithi140.front.sepia.ceph.com smithi140
2023-07-21T17:23:38.755 DEBUG:teuthology.orchestra.run.smithi140:> sudo hostname smithi140
2023-07-21T17:23:38.865 DEBUG:teuthology.orchestra.run.smithi140:> sudo sed -i -e 's/smithi140.front.sepia.ceph.com/smithi140/g' /etc/hosts
2023-07-21T17:23:38.894 DEBUG:teuthology.orchestra.run.smithi140:> sudo sed -i -e 's/172.21.15.140/172.21.15.140/g' /etc/hosts
2023-07-21T17:23:38.962 DEBUG:teuthology.orchestra.run.smithi140:> sudo sed -i -e 's/smithi140.front.sepia.ceph.com/smithi140/g' /etc/hostname
2023-07-21T17:23:39.027 DEBUG:teuthology.orchestra.run.smithi140:> sudo hostname smithi140
2023-07-21T17:23:39.093 DEBUG:teuthology.orchestra.run.smithi140:> cat /etc/os-release
2023-07-21T17:23:39.145 INFO:teuthology.orchestra.run.smithi140.stdout:NAME="CentOS Stream"
2023-07-21T17:23:39.146 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION="9"
2023-07-21T17:23:39.146 INFO:teuthology.orchestra.run.smithi140.stdout:ID="centos"
2023-07-21T17:23:39.146 INFO:teuthology.orchestra.run.smithi140.stdout:ID_LIKE="rhel fedora"
2023-07-21T17:23:39.146 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_ID="9"
2023-07-21T17:23:39.146 INFO:teuthology.orchestra.run.smithi140.stdout:PLATFORM_ID="platform:el9"
2023-07-21T17:23:39.146 INFO:teuthology.orchestra.run.smithi140.stdout:PRETTY_NAME="CentOS Stream 9"
2023-07-21T17:23:39.147 INFO:teuthology.orchestra.run.smithi140.stdout:ANSI_COLOR="0;31"
2023-07-21T17:23:39.147 INFO:teuthology.orchestra.run.smithi140.stdout:LOGO="fedora-logo-icon"
2023-07-21T17:23:39.147 INFO:teuthology.orchestra.run.smithi140.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-07-21T17:23:39.147 INFO:teuthology.orchestra.run.smithi140.stdout:HOME_URL="https://centos.org/"
2023-07-21T17:23:39.147 INFO:teuthology.orchestra.run.smithi140.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-21T17:23:39.147 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-07-21T17:23:39.147 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-21T17:23:39.148 INFO:teuthology.provision.fog.smithi140:Deploy complete!
2023-07-21T17:23:39.149 DEBUG:teuthology.parallel:result is None
2023-07-21T17:25:21.140 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-07-21T17:25:21.141 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi146
2023-07-21T17:25:30.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-07-21T17:25:30.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:25:35.341 DEBUG:teuthology.orchestra.run.smithi146:> true
2023-07-21T17:25:35.721 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-07-21T17:25:35.722 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-21T17:25:35.722 DEBUG:teuthology.orchestra.run.smithi146:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-21T17:25:35.736 INFO:teuthology.provision.fog.smithi146:Node is ready
2023-07-21T17:25:35.736 DEBUG:teuthology.orchestra.run.smithi146:> hostname
2023-07-21T17:25:35.791 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2023-07-21T17:25:35.792 DEBUG:teuthology.orchestra.run.smithi146:> grep smithi146.front.sepia.ceph.com /etc/hosts
2023-07-21T17:25:35.846 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2023-07-21T17:25:35.846 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146
2023-07-21T17:25:35.959 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hosts
2023-07-21T17:25:35.987 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/172.21.15.146/172.21.15.146/g' /etc/hosts
2023-07-21T17:25:36.053 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hostname
2023-07-21T17:25:36.121 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146
2023-07-21T17:25:36.187 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release
2023-07-21T17:25:36.242 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream"
2023-07-21T17:25:36.242 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9"
2023-07-21T17:25:36.242 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2023-07-21T17:25:36.242 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2023-07-21T17:25:36.243 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9"
2023-07-21T17:25:36.243 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9"
2023-07-21T17:25:36.243 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9"
2023-07-21T17:25:36.243 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2023-07-21T17:25:36.243 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon"
2023-07-21T17:25:36.243 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-07-21T17:25:36.243 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2023-07-21T17:25:36.244 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-21T17:25:36.244 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-07-21T17:25:36.244 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-21T17:25:36.244 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2023-07-21T17:25:36.244 DEBUG:teuthology.parallel:result is None
2023-07-21T17:25:46.439 INFO:teuthology.lock.ops:Checking smithi140.front.sepia.ceph.com
2023-07-21T17:25:46.440 INFO:teuthology.lock.ops:New key found. Updating...
2023-07-21T17:25:51.469 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2023-07-21T17:25:51.470 INFO:teuthology.lock.ops:New key found. Updating...
2023-07-21T17:25:53.637 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: set os type and version on server
2023-07-21T17:25:53.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi140.front.sepia.ceph.com'
2023-07-21T17:25:53.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:25:53.727 DEBUG:teuthology.orchestra.run.smithi140:> true
2023-07-21T17:25:53.898 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi140.front.sepia.ceph.com'
2023-07-21T17:25:53.899 DEBUG:teuthology.orchestra.run.smithi140:> uname -m
2023-07-21T17:25:53.953 INFO:teuthology.orchestra.run.smithi140.stdout:x86_64
2023-07-21T17:25:53.953 DEBUG:teuthology.orchestra.run.smithi140:> cat /etc/os-release
2023-07-21T17:25:54.007 INFO:teuthology.orchestra.run.smithi140.stdout:NAME="CentOS Stream"
2023-07-21T17:25:54.007 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION="9"
2023-07-21T17:25:54.007 INFO:teuthology.orchestra.run.smithi140.stdout:ID="centos"
2023-07-21T17:25:54.007 INFO:teuthology.orchestra.run.smithi140.stdout:ID_LIKE="rhel fedora"
2023-07-21T17:25:54.007 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_ID="9"
2023-07-21T17:25:54.007 INFO:teuthology.orchestra.run.smithi140.stdout:PLATFORM_ID="platform:el9"
2023-07-21T17:25:54.008 INFO:teuthology.orchestra.run.smithi140.stdout:PRETTY_NAME="CentOS Stream 9"
2023-07-21T17:25:54.008 INFO:teuthology.orchestra.run.smithi140.stdout:ANSI_COLOR="0;31"
2023-07-21T17:25:54.008 INFO:teuthology.orchestra.run.smithi140.stdout:LOGO="fedora-logo-icon"
2023-07-21T17:25:54.008 INFO:teuthology.orchestra.run.smithi140.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-07-21T17:25:54.008 INFO:teuthology.orchestra.run.smithi140.stdout:HOME_URL="https://centos.org/"
2023-07-21T17:25:54.008 INFO:teuthology.orchestra.run.smithi140.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-21T17:25:54.008 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-07-21T17:25:54.009 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-21T17:25:54.009 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server
2023-07-21T17:25:59.044 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2023-07-21T17:25:59.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-07-21T17:25:59.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T17:25:59.133 DEBUG:teuthology.orchestra.run.smithi146:> true
2023-07-21T17:25:59.306 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com'
2023-07-21T17:25:59.306 DEBUG:teuthology.orchestra.run.smithi146:> uname -m
2023-07-21T17:25:59.361 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2023-07-21T17:25:59.361 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release
2023-07-21T17:25:59.415 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream"
2023-07-21T17:25:59.415 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9"
2023-07-21T17:25:59.415 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2023-07-21T17:25:59.415 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2023-07-21T17:25:59.416 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9"
2023-07-21T17:25:59.416 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9"
2023-07-21T17:25:59.416 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9"
2023-07-21T17:25:59.416 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2023-07-21T17:25:59.416 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon"
2023-07-21T17:25:59.416 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-07-21T17:25:59.416 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2023-07-21T17:25:59.417 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-21T17:25:59.417 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-07-21T17:25:59.417 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-21T17:25:59.417 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2023-07-21T17:25:59.445 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T17:25:59.446 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T17:25:59.560 INFO:teuthology.dispatcher.supervisor:Running job 7346729
2023-07-21T17:25:59.561 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346729 --name yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi --description fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/iozone}}} -- /home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346729/orig.config.yaml
2023-07-21T17:25:59.568 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346729
2023-07-21T17:25:59.569 INFO:teuthology.dispatcher.supervisor:Job PID: 23155
2023-07-21T17:25:59.570 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-07-21T17:27:59.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T17:27:59.669 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T17:30:04.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T17:30:04.462 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T17:32:09.641 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T17:32:09.641 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T17:34:14.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T17:34:14.817 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T17:36:19.993 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T17:36:19.994 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T17:38:25.185 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T17:38:25.185 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T17:38:25.257 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-07-21T17:38:25.368 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-07-21T17:38:25.368 INFO:teuthology.nuke:Checking targets against current locks
2023-07-21T17:38:25.414 DEBUG:teuthology.nuke:shortname: smithi140
2023-07-21T17:38:25.415 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-21T17:38:25.434 DEBUG:teuthology.nuke:shortname: smithi146
2023-07-21T17:38:25.435 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-21T17:38:25.437 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346729', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 17:16:58.742358', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMuJurUuGN+cWDZSYKvCnTPf4pyCIb3kp4XlydTuBvRDGvG82xFaT/R0DD7Wk9TYFl71RRbCSb5nplg+4n6oP1s='}
2023-07-21T17:38:25.461 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346729', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 17:16:58.744416', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJkgKQDGYNBsRBWm6vmbaxI28qIPQW0gHWBndtZKvGvkDchfZeByvXTaZZvD2cEi3HvE/XkGXFlnMPyrgJpZgbU='}
2023-07-21T17:38:25.485 INFO:teuthology.orchestra.console:Power off smithi140
2023-07-21T17:38:25.485 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-21T17:38:25.508 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:38:25.526 INFO:teuthology.orchestra.console:Power off smithi146
2023-07-21T17:38:25.527 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-21T17:38:25.548 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T17:38:25.638 INFO:teuthology.orchestra.console:Power off for smithi140 completed
2023-07-21T17:38:25.674 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2023-07-21T17:38:25.824 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com
2023-07-21T17:38:25.833 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com