2023-07-21T11:32:00.632 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T11:32:00.634 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:32:00.634 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:32:00.802 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fba449768b0>, signals=[15]) 2023-07-21T11:32:00.805 INFO:teuthology.lock.ops:Start node 'smithi136.front.sepia.ceph.com' reimaging 2023-07-21T11:32:00.805 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T11:32:00.805 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2023-07-21T11:32:00.833 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging 2023-07-21T11:32:00.834 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T11:32:00.834 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-07-21T11:32:00.835 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /host {"name": "smithi136"} 2023-07-21T11:32:15.865 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /host {"name": "smithi154"} 2023-07-21T11:32:20.896 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-21T11:32:30.935 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-21T11:32:31.998 DEBUG:teuthology.provision.fog.smithi136:Requesting image smithi_centos_9.stream (ID 10) 2023-07-21T11:32:31.999 DEBUG:teuthology.provision.fog.smithi136:FOG request: PUT /host/241 {"imageID": 10} 2023-07-21T11:32:32.042 DEBUG:teuthology.provision.fog.smithi154:Requesting image smithi_centos_9.stream (ID 10) 2023-07-21T11:32:32.042 DEBUG:teuthology.provision.fog.smithi154:FOG request: PUT /host/259 {"imageID": 10} 2023-07-21T11:32:32.049 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of centos 9.stream 2023-07-21T11:32:32.050 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:32:32.087 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of centos 9.stream 2023-07-21T11:32:32.087 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None 2023-07-21T11:32:32.105 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T11:32:32.136 DEBUG:teuthology.provision.fog.smithi136:FOG request: POST /host/241/task {"taskTypeID": 1} 2023-07-21T11:32:32.141 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T11:32:32.179 DEBUG:teuthology.provision.fog.smithi154:FOG request: POST /host/259/task {"taskTypeID": 1} 2023-07-21T11:32:32.188 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:32:32.229 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None 2023-07-21T11:32:42.299 INFO:teuthology.orchestra.console:Power off smithi154 2023-07-21T11:32:42.300 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T11:32:42.315 INFO:teuthology.orchestra.console:Power off smithi136 2023-07-21T11:32:42.316 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T11:32:42.327 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:42.339 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:42.452 INFO:teuthology.orchestra.console:Power off for smithi154 completed 2023-07-21T11:32:42.465 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2023-07-21T11:32:42.553 INFO:teuthology.orchestra.console:Power on smithi154 2023-07-21T11:32:42.554 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T11:32:42.566 INFO:teuthology.orchestra.console:Power on smithi136 2023-07-21T11:32:42.566 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T11:32:42.576 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:42.588 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:46.600 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:46.611 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:51.713 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:51.724 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:56.828 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:56.838 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T11:32:57.050 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2023-07-21T11:32:57.151 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2023-07-21T11:32:57.151 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:33:02.258 INFO:teuthology.orchestra.console:Power on for smithi154 completed 2023-07-21T11:33:02.358 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish 2023-07-21T11:33:02.359 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None 2023-07-21T11:33:07.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:33:15.472 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-07-21T11:33:17.212 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:33:22.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:33:30.288 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-07-21T11:33:36.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:33:42.276 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:33:47.282 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-07-21T11:33:53.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:34:01.360 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-07-21T11:34:02.340 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:34:07.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:34:15.440 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-07-21T11:34:17.399 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:34:22.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:34:29.692 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-07-21T11:34:35.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:34:40.780 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed. 2023-07-21T11:34:40.782 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T11:34:40.783 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T11:34:40.784 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:41.788 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:42.463 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:34:42.794 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:43.799 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:44.804 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:45.809 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:46.813 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:47.818 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:48.823 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:49.829 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:50.835 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:51.841 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:52.846 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:53.852 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:54.857 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:55.862 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:56.867 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:57.522 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:34:57.873 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:58.879 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:34:59.884 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:00.889 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:01.894 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:02.899 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:03.904 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:04.910 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:05.915 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:06.921 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:07.926 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:08.932 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:09.940 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:10.946 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:11.952 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:12.572 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:35:17.577 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:18.583 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:19.588 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:20.593 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:21.598 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:22.602 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:23.607 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T11:35:32.627 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-21T11:35:37.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:35:40.752 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-21T11:35:46.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:35:49.808 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-21T11:35:55.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:36:03.888 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-21T11:36:09.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:36:17.968 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-21T11:36:23.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:36:47.472 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-21T11:36:53.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:36:56.528 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-21T11:37:02.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:37:02.776 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T11:37:02.776 DEBUG:teuthology.orchestra.run.smithi136:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T11:37:11.540 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-21T11:37:11.541 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi154 2023-07-21T11:37:20.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-07-21T11:37:20.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:37:21.010 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-07-21T11:37:21.344 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-07-21T11:37:21.344 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T11:37:21.345 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T11:37:23.377 INFO:teuthology.provision.fog.smithi136:Node is ready 2023-07-21T11:37:23.380 DEBUG:teuthology.orchestra.run.smithi136:> hostname 2023-07-21T11:37:23.399 INFO:teuthology.orchestra.run.smithi136.stdout:smithi197 2023-07-21T11:37:23.399 DEBUG:teuthology.orchestra.run.smithi136:> grep smithi197 /etc/hosts 2023-07-21T11:37:23.457 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197 2023-07-21T11:37:23.458 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname smithi136 2023-07-21T11:37:23.578 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/smithi197/smithi136/g' /etc/hosts 2023-07-21T11:37:23.610 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/172.21.15.197/172.21.15.136/g' /etc/hosts 2023-07-21T11:37:23.681 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/smithi197/smithi136/g' /etc/hostname 2023-07-21T11:37:23.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname smithi136 2023-07-21T11:37:23.817 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2023-07-21T11:37:23.872 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2023-07-21T11:37:23.872 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2023-07-21T11:37:23.873 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2023-07-21T11:37:23.873 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2023-07-21T11:37:23.873 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2023-07-21T11:37:23.873 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2023-07-21T11:37:23.874 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T11:37:23.874 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2023-07-21T11:37:23.874 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2023-07-21T11:37:23.875 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T11:37:23.875 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2023-07-21T11:37:23.875 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T11:37:23.875 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T11:37:23.876 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T11:37:23.877 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2023-07-21T11:37:23.878 DEBUG:teuthology.parallel:result is None 2023-07-21T11:37:31.362 INFO:teuthology.provision.fog.smithi154:Node is ready 2023-07-21T11:37:31.362 DEBUG:teuthology.orchestra.run.smithi154:> hostname 2023-07-21T11:37:31.380 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com 2023-07-21T11:37:31.381 DEBUG:teuthology.orchestra.run.smithi154:> grep smithi154.front.sepia.ceph.com /etc/hosts 2023-07-21T11:37:31.441 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154 2023-07-21T11:37:31.442 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154 2023-07-21T11:37:31.574 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hosts 2023-07-21T11:37:31.605 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/172.21.15.154/172.21.15.154/g' /etc/hosts 2023-07-21T11:37:31.675 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hostname 2023-07-21T11:37:31.743 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154 2023-07-21T11:37:31.811 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-07-21T11:37:31.866 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2023-07-21T11:37:31.867 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2023-07-21T11:37:31.867 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2023-07-21T11:37:31.868 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2023-07-21T11:37:31.868 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2023-07-21T11:37:31.868 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2023-07-21T11:37:31.869 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T11:37:31.869 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2023-07-21T11:37:31.869 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2023-07-21T11:37:31.870 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T11:37:31.870 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2023-07-21T11:37:31.870 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T11:37:31.870 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T11:37:31.871 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T11:37:31.871 INFO:teuthology.provision.fog.smithi154:Deploy complete! 2023-07-21T11:37:31.872 DEBUG:teuthology.parallel:result is None 2023-07-21T11:37:32.916 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2023-07-21T11:37:32.920 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T11:37:32.948 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com 2023-07-21T11:37:32.949 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T11:37:32.974 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2023-07-21T11:37:32.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-07-21T11:37:32.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:37:33.060 DEBUG:teuthology.orchestra.run.smithi136:> true 2023-07-21T11:37:33.133 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-07-21T11:37:33.133 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2023-07-21T11:37:33.188 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2023-07-21T11:37:33.189 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2023-07-21T11:37:33.244 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2023-07-21T11:37:33.245 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2023-07-21T11:37:33.245 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2023-07-21T11:37:33.245 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2023-07-21T11:37:33.245 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2023-07-21T11:37:33.245 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2023-07-21T11:37:33.245 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T11:37:33.245 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2023-07-21T11:37:33.246 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2023-07-21T11:37:33.246 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T11:37:33.246 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2023-07-21T11:37:33.246 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T11:37:33.246 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T11:37:33.246 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T11:37:33.247 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2023-07-21T11:37:33.273 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server 2023-07-21T11:37:33.274 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-07-21T11:37:33.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T11:37:33.355 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-07-21T11:37:33.424 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-07-21T11:37:33.424 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2023-07-21T11:37:33.480 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2023-07-21T11:37:33.481 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-07-21T11:37:33.536 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2023-07-21T11:37:33.537 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2023-07-21T11:37:33.537 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2023-07-21T11:37:33.537 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2023-07-21T11:37:33.537 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2023-07-21T11:37:33.537 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2023-07-21T11:37:33.537 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T11:37:33.537 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2023-07-21T11:37:33.538 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2023-07-21T11:37:33.538 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T11:37:33.538 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2023-07-21T11:37:33.538 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T11:37:33.538 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T11:37:33.538 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T11:37:33.539 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-07-21T11:37:33.565 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:37:33.566 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:37:33.677 INFO:teuthology.dispatcher.supervisor:Running job 7346090 2023-07-21T11:37:33.678 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-21_02:03:58-rados-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346090 --name yuriw-2023-07-21_02:03:58-rados-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi --description rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} -- /home/teuthworker/archive/yuriw-2023-07-21_02:03:58-rados-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346090/orig.config.yaml 2023-07-21T11:37:33.681 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-21_02:03:58-rados-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346090 2023-07-21T11:37:33.681 INFO:teuthology.dispatcher.supervisor:Job PID: 21591 2023-07-21T11:37:33.682 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-21T11:39:33.780 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:39:33.781 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:41:38.173 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:41:38.174 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:43:39.689 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:43:39.690 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:45:39.861 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:45:39.862 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:47:40.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:47:40.042 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:49:40.237 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:49:40.238 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:51:44.414 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:51:44.415 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:53:44.585 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:53:44.586 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:55:44.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:55:44.706 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:57:44.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:57:44.905 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T11:59:45.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T11:59:45.069 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T12:01:45.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T12:01:45.254 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T12:01:45.339 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-21T12:01:45.435 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-21T12:01:45.435 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T12:01:45.487 DEBUG:teuthology.nuke:shortname: smithi136 2023-07-21T12:01:45.487 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T12:01:45.513 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-21_02:03:58-rados-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346090', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 11:31:59.788012', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWul9pRc6YWv5814MRufGgSr5BsblNeYBcCH0wvbkGodjQBz+PNAn39OtWdg2Rs2EYHWcAoGvZ7UODyY4mpOd0='} 2023-07-21T12:01:45.518 DEBUG:teuthology.nuke:shortname: smithi154 2023-07-21T12:01:45.518 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T12:01:45.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-21_02:03:58-rados-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346090', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 11:31:59.789561', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFPejcb/SUKPTDdFZ406FQI1ZAturTXy7wMZiYgsQ0l/yFyVpchBIhr7ZEDsbve9cxoeTYaIqvqncCEDtHrPlxM='} 2023-07-21T12:01:45.583 INFO:teuthology.orchestra.console:Power off smithi136 2023-07-21T12:01:45.584 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T12:01:45.610 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T12:01:45.632 INFO:teuthology.orchestra.console:Power off smithi154 2023-07-21T12:01:45.633 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T12:01:45.656 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T12:01:45.744 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2023-07-21T12:01:45.779 INFO:teuthology.orchestra.console:Power off for smithi154 completed 2023-07-21T12:01:45.904 INFO:teuthology.lock.ops:unlocked: smithi136.front.sepia.ceph.com 2023-07-21T12:01:45.929 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com