2023-06-22T14:50:47.928 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb
2023-06-22T14:50:47.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T14:50:47.930 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T14:51:13.093 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f2647081730>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f2646a244c0>, signals=[15])
2023-06-22T14:51:13.095 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging
2023-06-22T14:51:13.095 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server
2023-06-22T14:51:13.095 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server
2023-06-22T14:51:18.130 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging
2023-06-22T14:51:18.131 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server
2023-06-22T14:51:18.131 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2023-06-22T14:51:18.132 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /host {"name": "smithi033"}
2023-06-22T14:51:28.161 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /host {"name": "smithi149"}
2023-06-22T14:51:38.224 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-22T14:51:43.230 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-22T14:51:46.038 DEBUG:teuthology.provision.fog.smithi033:Requesting image smithi_centos_8.stream (ID 3)
2023-06-22T14:51:46.039 DEBUG:teuthology.provision.fog.smithi033:FOG request: PUT /host/138 {"imageID": 3}
2023-06-22T14:51:46.041 DEBUG:teuthology.provision.fog.smithi149:Requesting image smithi_centos_8.stream (ID 3)
2023-06-22T14:51:46.041 DEBUG:teuthology.provision.fog.smithi149:FOG request: PUT /host/254 {"imageID": 3}
2023-06-22T14:51:46.080 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of centos 8.stream
2023-06-22T14:51:46.080 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:51:46.083 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of centos 8.stream
2023-06-22T14:51:46.083 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None
2023-06-22T14:51:46.123 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /tasktype {"name": "deploy"}
2023-06-22T14:51:46.126 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /tasktype {"name": "deploy"}
2023-06-22T14:51:46.159 DEBUG:teuthology.provision.fog.smithi149:FOG request: POST /host/254/task {"taskTypeID": 1}
2023-06-22T14:51:46.164 DEBUG:teuthology.provision.fog.smithi033:FOG request: POST /host/138/task {"taskTypeID": 1}
2023-06-22T14:51:46.293 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:51:46.302 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None
2023-06-22T14:51:56.373 INFO:teuthology.orchestra.console:Power off smithi149
2023-06-22T14:51:56.374 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-22T14:51:56.389 INFO:teuthology.orchestra.console:Power off smithi033
2023-06-22T14:51:56.390 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-22T14:52:01.399 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:01.415 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:01.525 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2023-06-22T14:52:01.627 INFO:teuthology.orchestra.console:Power on smithi149
2023-06-22T14:52:01.627 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-22T14:52:01.650 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:05.676 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:06.425 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:09.790 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:11.537 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:14.900 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:16.647 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:16.858 INFO:teuthology.orchestra.console:Power off for smithi033 completed
2023-06-22T14:52:16.959 INFO:teuthology.orchestra.console:Power on smithi033
2023-06-22T14:52:16.960 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-22T14:52:16.982 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:20.012 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:21.008 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:21.218 INFO:teuthology.orchestra.console:Power on for smithi033 completed
2023-06-22T14:52:21.320 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish
2023-06-22T14:52:21.321 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None
2023-06-22T14:52:26.329 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T14:52:26.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:52:31.497 INFO:teuthology.orchestra.console:Power on for smithi149 completed
2023-06-22T14:52:31.597 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish
2023-06-22T14:52:31.598 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:52:49.904 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-06-22T14:52:51.304 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:52:55.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:52:58.960 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-06-22T14:53:04.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:53:06.373 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:53:08.016 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-06-22T14:53:14.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:53:21.440 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:53:32.400 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-06-22T14:53:38.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:53:40.855 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-06-22T14:53:41.506 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:53:46.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:53:46.982 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:53:53.117 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:53:59.123 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:54:04.245 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:54:10.367 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:54:16.490 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:54:22.496 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:54:27.572 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:54:31.615 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:54:32.734 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:54:33.851 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:54:34.969 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-06-22T14:54:34.970 WARNING:teuthology.provision.fog.smithi033:'connect to smithi033.front.sepia.ceph.com' reached maximum tries (11) after waiting for 10 seconds
2023-06-22T14:54:40.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:54:45.977 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-22T14:54:51.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:55:14.544 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-22T14:55:20.436 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-06-22T14:55:20.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:55:26.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:55:28.624 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-22T14:55:29.488 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-06-22T14:55:34.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:55:35.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:55:38.800 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-22T14:55:44.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:55:47.856 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-22T14:55:53.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:56:35.750 WARNING:teuthology.provision.fog.smithi033:timed out
2023-06-22T14:56:41.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:56:41.998 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-22T14:56:41.999 DEBUG:teuthology.orchestra.run.smithi033:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-22T14:56:42.274 INFO:teuthology.provision.fog.smithi033:Node is ready
2023-06-22T14:56:42.275 DEBUG:teuthology.orchestra.run.smithi033:> hostname
2023-06-22T14:56:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com
2023-06-22T14:56:42.297 DEBUG:teuthology.orchestra.run.smithi033:> grep smithi033.front.sepia.ceph.com /etc/hosts
2023-06-22T14:56:42.358 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033
2023-06-22T14:56:42.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033
2023-06-22T14:56:42.472 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hosts
2023-06-22T14:56:42.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/172.21.15.33/172.21.15.33/g' /etc/hosts
2023-06-22T14:56:42.577 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hostname
2023-06-22T14:56:42.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033
2023-06-22T14:56:42.718 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release
2023-06-22T14:56:42.778 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream"
2023-06-22T14:56:42.778 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8"
2023-06-22T14:56:42.779 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos"
2023-06-22T14:56:42.779 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora"
2023-06-22T14:56:42.779 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8"
2023-06-22T14:56:42.779 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8"
2023-06-22T14:56:42.779 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-22T14:56:42.779 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31"
2023-06-22T14:56:42.779 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-22T14:56:42.780 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/"
2023-06-22T14:56:42.780 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-22T14:56:42.780 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-22T14:56:42.780 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-22T14:56:42.781 INFO:teuthology.provision.fog.smithi033:Deploy complete!
2023-06-22T14:56:42.783 DEBUG:teuthology.parallel:result is None
2023-06-22T14:56:53.860 WARNING:teuthology.provision.fog.smithi149:timed out
2023-06-22T14:56:59.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:57:00.083 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-22T14:57:00.083 DEBUG:teuthology.orchestra.run.smithi149:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-22T14:57:00.343 INFO:teuthology.provision.fog.smithi149:Node is ready
2023-06-22T14:57:00.344 DEBUG:teuthology.orchestra.run.smithi149:> hostname
2023-06-22T14:57:00.364 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com
2023-06-22T14:57:00.364 DEBUG:teuthology.orchestra.run.smithi149:> grep smithi149.front.sepia.ceph.com /etc/hosts
2023-06-22T14:57:00.424 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149
2023-06-22T14:57:00.424 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149
2023-06-22T14:57:00.561 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149.front.sepia.ceph.com/smithi149/g' /etc/hosts
2023-06-22T14:57:00.592 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/172.21.15.149/172.21.15.149/g' /etc/hosts
2023-06-22T14:57:00.661 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149.front.sepia.ceph.com/smithi149/g' /etc/hostname
2023-06-22T14:57:00.730 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149
2023-06-22T14:57:00.798 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release
2023-06-22T14:57:00.856 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream"
2023-06-22T14:57:00.857 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8"
2023-06-22T14:57:00.857 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos"
2023-06-22T14:57:00.857 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora"
2023-06-22T14:57:00.857 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8"
2023-06-22T14:57:00.857 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8"
2023-06-22T14:57:00.857 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-22T14:57:00.858 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2023-06-22T14:57:00.858 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-22T14:57:00.858 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/"
2023-06-22T14:57:00.858 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-22T14:57:00.859 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-22T14:57:00.859 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-22T14:57:00.859 INFO:teuthology.provision.fog.smithi149:Deploy complete!
2023-06-22T14:57:00.860 DEBUG:teuthology.parallel:result is None
2023-06-22T14:57:36.073 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com
2023-06-22T14:57:36.074 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-22T14:57:41.115 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com
2023-06-22T14:57:41.116 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-22T14:57:46.158 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server
2023-06-22T14:57:46.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com'
2023-06-22T14:57:46.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:57:51.234 DEBUG:teuthology.orchestra.run.smithi033:> true
2023-06-22T14:57:51.311 DEBUG:teuthology.orchestra.run.smithi033:> uname -m
2023-06-22T14:57:51.373 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64
2023-06-22T14:57:51.374 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release
2023-06-22T14:57:51.434 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream"
2023-06-22T14:57:51.434 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8"
2023-06-22T14:57:51.434 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos"
2023-06-22T14:57:51.435 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora"
2023-06-22T14:57:51.435 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8"
2023-06-22T14:57:51.435 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8"
2023-06-22T14:57:51.435 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-22T14:57:51.435 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31"
2023-06-22T14:57:51.435 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-22T14:57:51.435 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/"
2023-06-22T14:57:51.436 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-22T14:57:51.436 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-22T14:57:51.436 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-22T14:57:51.436 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server
2023-06-22T14:57:56.471 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server
2023-06-22T14:57:56.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2023-06-22T14:57:56.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-22T14:57:56.537 DEBUG:teuthology.orchestra.run.smithi149:> true
2023-06-22T14:57:56.672 DEBUG:teuthology.orchestra.run.smithi149:> uname -m
2023-06-22T14:57:56.731 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64
2023-06-22T14:57:56.732 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release
2023-06-22T14:57:56.791 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream"
2023-06-22T14:57:56.791 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8"
2023-06-22T14:57:56.791 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos"
2023-06-22T14:57:56.791 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora"
2023-06-22T14:57:56.791 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8"
2023-06-22T14:57:56.792 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8"
2023-06-22T14:57:56.792 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-22T14:57:56.792 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2023-06-22T14:57:56.792 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-22T14:57:56.792 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/"
2023-06-22T14:57:56.792 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-22T14:57:56.792 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-22T14:57:56.793 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-22T14:57:56.793 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2023-06-22T14:58:01.820 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T14:58:01.821 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T14:58:02.549 INFO:teuthology.dispatcher.supervisor:Running job 7311507
2023-06-22T14:58:02.549 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-06-22_11:20:21-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7311507 --name rishabh-2023-06-22_11:20:21-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi --description fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} -- /home/teuthworker/archive/rishabh-2023-06-22_11:20:21-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7311507/orig.config.yaml
2023-06-22T14:58:02.554 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-06-22_11:20:21-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7311507
2023-06-22T14:58:02.555 INFO:teuthology.dispatcher.supervisor:Job PID: 18644
2023-06-22T14:58:02.555 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-06-22T15:00:02.556 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:00:02.557 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:02:02.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:02:02.633 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:04:07.720 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:04:07.721 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:06:10.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:06:10.070 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:08:13.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:08:13.226 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:10:17.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:10:17.118 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:12:22.190 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:12:22.191 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:14:27.270 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:14:27.271 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:16:27.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:16:27.348 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:18:32.418 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:18:32.420 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:20:37.492 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:20:37.493 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:22:42.568 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:22:42.569 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:24:47.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:24:47.647 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:26:52.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-22T15:26:52.731 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-22T15:26:57.569 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-06-22T15:26:57.669 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-06-22T15:26:57.669 INFO:teuthology.nuke:Checking targets against current locks
2023-06-22T15:27:07.731 DEBUG:teuthology.nuke:shortname: smithi033
2023-06-22T15:27:07.732 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-22T15:27:17.773 DEBUG:teuthology.nuke:shortname: smithi149
2023-06-22T15:27:17.774 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-22T15:27:22.784 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-22_11:20:21-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7311507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 14:50:32.102005', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnejMoXkwH84jXeJWE0Qtt+r9zIKVthHRAvUBYFB9GG1EgiovAorGbBdvKmLgUOHVZ7SNcEv3z463f2oFw0YBl4JKsMBlqo5PqEVF04GboKUJZphvIpMQmRSAsBk5XASv/vH4Bnx/olUtFsWoQnkac46Thagfqh0b2+4NQuL2JK0UYRI6O56yYqMrXzGmGttoC0kGoHtbx2b84HSmmZq/F5O2Wt0DXtORFzTxUfeI6A5/mfRQYU7O+OrSJGaXNoXFyCu5JfhSIjMH9dGeXaGGhx+cw7aS9UzBnbi68agdyTkjUKI427VY0vYm/ThPf3PKqK6rx6qWkeFp0cNm89HI6HKg6GYDF37qMl57xjPIbWnJFsMonV0D4r8DgJQuGj5JqFfgK2BUJ1HdR9mPiNWoe1Rp8ZUcA13os4cJTwSnbj5+u19QonOFSfo9y9fGGafPs2iivA6BZd2UZk6KH8UX/doITDtr9oZ23Xsjx0RWMg0rxWI0wQzFNwHtKK4DI51E='}
2023-06-22T15:27:27.815 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-22_11:20:21-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7311507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 14:50:32.100767', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsPgyUA2diHWDrZ4GSkwmzQtI7UxEoWrv2VgHyBxQEGxp21elUnDvbXOn6Sbinh/xHhj9GjW/VbCpuhH8BN9QE3cUlIcfw31RPyBRbqdssY+kl5YTC2gA3xnZdyhRgNrY62+VsQr0gEOVZSlGMxH4EHSPjCZxJ7ts9A8LUaNY6vdmI8Q0hXedWZ+FalTjmRj2GYAzKYfk1Cr69EQfFnAcWyz83iF3n8jai9DdWh5Za8Ot/vgCuL5j5fox9Zydfpk0ye4Ufq5TY4dtBldHcDTVHioOHXU6Ipycuhs9ePN9nv6NSea4w5grl07lHBnBCDIFBFBWWODq5Llnhx6W4/4Bz13QB4DLhS+v9fhglfrbKcY7k0+Nucqga5NPijL7syke7GsMsb9Nu8u5RONbrgiAxyZCl3boYVGrW9FPjSJpkAKKrqQNmXK7HBWtEvP1mNOWDMCTlMyzboGWGAP36vQhxxB8BOUYKzHkD9PZBecF9d0EE8dl444nxbNEfy0DE2PU='}
2023-06-22T15:27:37.834 INFO:teuthology.orchestra.console:Power off smithi033
2023-06-22T15:27:37.835 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-22T15:27:38.868 INFO:teuthology.orchestra.console:Power off smithi149
2023-06-22T15:27:38.869 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-22T15:27:42.859 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:27:43.894 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:27:47.873 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:27:48.906 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:27:53.040 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:27:54.022 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:27:58.152 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:27:58.366 INFO:teuthology.orchestra.console:Power off for smithi033 completed
2023-06-22T15:27:58.975 INFO:teuthology.lock.ops:unlocked: smithi033.front.sepia.ceph.com
2023-06-22T15:27:59.135 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:28:04.245 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:28:09.358 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-22T15:28:09.570 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2023-06-22T15:28:19.739 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com