2023-08-08T01:15:37.442 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-08T01:15:37.443 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:15:37.585 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcbf6158f70>, signals=[15]) 2023-08-08T01:15:37.586 INFO:teuthology.lock.ops:Start node 'smithi039.front.sepia.ceph.com' reimaging 2023-08-08T01:15:37.587 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: reset os type and version on server 2023-08-08T01:15:37.587 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2023-08-08T01:15:37.608 INFO:teuthology.lock.ops:Start node 'smithi053.front.sepia.ceph.com' reimaging 2023-08-08T01:15:37.609 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: reset os type and version on server 2023-08-08T01:15:37.609 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-08-08T01:15:37.610 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /host {"name": "smithi039"} 2023-08-08T01:15:37.633 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /host {"name": "smithi053"} 2023-08-08T01:15:37.677 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-08-08T01:15:37.700 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-08-08T01:15:37.716 DEBUG:teuthology.provision.fog.smithi039:Requesting image smithi_centos_9.stream (ID 10) 2023-08-08T01:15:37.717 DEBUG:teuthology.provision.fog.smithi039:FOG request: PUT /host/144 {"imageID": 10} 2023-08-08T01:15:37.745 DEBUG:teuthology.provision.fog.smithi053:Requesting image smithi_centos_9.stream (ID 10) 2023-08-08T01:15:37.746 DEBUG:teuthology.provision.fog.smithi053:FOG request: PUT /host/158 {"imageID": 10} 2023-08-08T01:15:37.761 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of centos 9.stream 2023-08-08T01:15:37.761 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:15:37.792 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of centos 9.stream 2023-08-08T01:15:37.792 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:15:37.817 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /tasktype {"name": "deploy"} 2023-08-08T01:15:37.840 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /tasktype {"name": "deploy"} 2023-08-08T01:15:37.859 DEBUG:teuthology.provision.fog.smithi039:FOG request: POST /host/144/task {"taskTypeID": 1} 2023-08-08T01:15:37.882 DEBUG:teuthology.provision.fog.smithi053:FOG request: POST /host/158/task {"taskTypeID": 1} 2023-08-08T01:15:37.930 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:15:37.949 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:15:38.005 INFO:teuthology.orchestra.console.smithi039:Power off 2023-08-08T01:15:38.006 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T01:15:38.021 INFO:teuthology.orchestra.console.smithi053:Power off 2023-08-08T01:15:38.022 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T01:15:38.032 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2023-08-08T01:15:38.033 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:38.044 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2023-08-08T01:15:38.045 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:38.057 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-08-08T01:15:38.068 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-08-08T01:15:42.060 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:42.070 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:42.170 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-08-08T01:15:42.179 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-08-08T01:15:46.176 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:46.185 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:46.287 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-08-08T01:15:46.296 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-08-08T01:15:46.397 INFO:teuthology.orchestra.console.smithi053:Power off completed 2023-08-08T01:15:46.497 INFO:teuthology.orchestra.console.smithi053:Power on 2023-08-08T01:15:46.498 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-08T01:15:46.520 DEBUG:teuthology.orchestra.console.smithi053:power on output: Chassis Power Control: Up/On 2023-08-08T01:15:46.521 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:46.541 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-08-08T01:15:50.292 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:50.401 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2023-08-08T01:15:50.502 INFO:teuthology.orchestra.console.smithi039:Power off completed 2023-08-08T01:15:50.543 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:50.604 INFO:teuthology.orchestra.console.smithi039:Power on 2023-08-08T01:15:50.605 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-08T01:15:50.628 DEBUG:teuthology.orchestra.console.smithi039:power on output: Chassis Power Control: Up/On 2023-08-08T01:15:50.629 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:50.650 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2023-08-08T01:15:50.652 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-08-08T01:15:54.656 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:54.663 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:15:54.766 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-08-08T01:15:54.773 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-08-08T01:15:54.868 INFO:teuthology.orchestra.console.smithi039:Power on completed 2023-08-08T01:15:54.874 INFO:teuthology.orchestra.console.smithi053:Power on completed 2023-08-08T01:15:54.969 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish 2023-08-08T01:15:54.970 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:15:54.976 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish 2023-08-08T01:15:54.976 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:16:10.036 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:16:10.038 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:16:25.100 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:16:25.102 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:16:40.164 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:16:40.165 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:16:55.228 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:16:55.229 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:17:10.291 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:17:10.292 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:17:25.352 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:17:25.353 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:17:40.424 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:17:40.426 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:17:55.496 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:17:55.497 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:18:10.568 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-08-08T01:18:10.569 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:18:10.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:18:13.716 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-08-08T01:18:19.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:18:22.800 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-08-08T01:18:25.636 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:18:28.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:18:31.856 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-08-08T01:18:37.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:18:40.696 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:18:40.916 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-08-08T01:18:46.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:18:50.004 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-08-08T01:18:55.757 DEBUG:teuthology.provision.fog.smithi039:FOG request: GET /task/active None 2023-08-08T01:18:55.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:18:55.917 ERROR:teuthology.orchestra.connection:Error authenticating with smithi039.front.sepia.ceph.com: Authentication failed. 2023-08-08T01:18:55.919 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-08T01:18:55.920 DEBUG:teuthology.orchestra.run.smithi039:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-08T01:18:55.921 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-08T01:18:56.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:18:56.927 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-08T01:18:57.935 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-08T01:18:58.941 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-08T01:19:56.008 WARNING:teuthology.provision.fog.smithi053:timed out 2023-08-08T01:20:02.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:20:02.213 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-08T01:20:02.214 DEBUG:teuthology.orchestra.run.smithi053:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-08T01:20:02.546 INFO:teuthology.provision.fog.smithi053:Node is ready 2023-08-08T01:20:02.547 DEBUG:teuthology.orchestra.run.smithi053:> hostname 2023-08-08T01:20:02.563 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053.front.sepia.ceph.com 2023-08-08T01:20:02.564 DEBUG:teuthology.orchestra.run.smithi053:> grep smithi053.front.sepia.ceph.com /etc/hosts 2023-08-08T01:20:02.621 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53 smithi053.front.sepia.ceph.com smithi053 2023-08-08T01:20:02.622 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053 2023-08-08T01:20:02.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hosts 2023-08-08T01:20:02.770 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/172.21.15.53/172.21.15.53/g' /etc/hosts 2023-08-08T01:20:02.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hostname 2023-08-08T01:20:02.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053 2023-08-08T01:20:02.984 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-08-08T01:20:03.039 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2023-08-08T01:20:03.040 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2023-08-08T01:20:03.040 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2023-08-08T01:20:03.040 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2023-08-08T01:20:03.040 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2023-08-08T01:20:03.040 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2023-08-08T01:20:03.040 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-08T01:20:03.041 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-08-08T01:20:03.041 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2023-08-08T01:20:03.041 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-08T01:20:03.041 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2023-08-08T01:20:03.041 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-08T01:20:03.041 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-08T01:20:03.042 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-08T01:20:03.042 INFO:teuthology.provision.fog.smithi053:Deploy complete! 2023-08-08T01:20:03.043 DEBUG:teuthology.parallel:result is None 2023-08-08T01:21:46.101 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-08-08T01:21:46.102 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi039 2023-08-08T01:21:55.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-08-08T01:21:55.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:21:55.319 DEBUG:teuthology.orchestra.run.smithi039:> true 2023-08-08T01:21:55.655 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-08-08T01:21:55.655 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-08T01:21:55.656 DEBUG:teuthology.orchestra.run.smithi039:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-08T01:21:55.671 INFO:teuthology.provision.fog.smithi039:Node is ready 2023-08-08T01:21:55.671 DEBUG:teuthology.orchestra.run.smithi039:> hostname 2023-08-08T01:21:55.727 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 2023-08-08T01:21:55.728 DEBUG:teuthology.orchestra.run.smithi039:> grep smithi039 /etc/hosts 2023-08-08T01:21:55.782 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39 smithi039.front.sepia.ceph.com smithi039 2023-08-08T01:21:55.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname smithi039 2023-08-08T01:21:55.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/smithi039/smithi039/g' /etc/hosts 2023-08-08T01:21:55.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/172.21.15.39/172.21.15.39/g' /etc/hosts 2023-08-08T01:21:56.000 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/smithi039/smithi039/g' /etc/hostname 2023-08-08T01:21:56.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname smithi039 2023-08-08T01:21:56.143 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2023-08-08T01:21:56.198 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2023-08-08T01:21:56.198 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2023-08-08T01:21:56.198 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2023-08-08T01:21:56.198 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2023-08-08T01:21:56.198 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2023-08-08T01:21:56.198 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2023-08-08T01:21:56.199 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-08T01:21:56.199 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2023-08-08T01:21:56.199 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2023-08-08T01:21:56.199 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-08T01:21:56.199 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2023-08-08T01:21:56.199 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-08T01:21:56.199 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-08T01:21:56.200 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-08T01:21:56.200 INFO:teuthology.provision.fog.smithi039:Deploy complete! 2023-08-08T01:21:56.200 DEBUG:teuthology.parallel:result is None 2023-08-08T01:21:56.375 INFO:teuthology.lock.ops:Checking smithi039.front.sepia.ceph.com 2023-08-08T01:21:56.376 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-08T01:21:56.395 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com 2023-08-08T01:21:56.396 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-08T01:21:56.413 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: set os type and version on server 2023-08-08T01:21:56.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-08-08T01:21:56.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:21:56.510 DEBUG:teuthology.orchestra.run.smithi039:> true 2023-08-08T01:21:56.578 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-08-08T01:21:56.579 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2023-08-08T01:21:56.634 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2023-08-08T01:21:56.634 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2023-08-08T01:21:56.688 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2023-08-08T01:21:56.688 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2023-08-08T01:21:56.689 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2023-08-08T01:21:56.689 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2023-08-08T01:21:56.689 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2023-08-08T01:21:56.689 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2023-08-08T01:21:56.689 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-08T01:21:56.689 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2023-08-08T01:21:56.690 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2023-08-08T01:21:56.690 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-08T01:21:56.690 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2023-08-08T01:21:56.690 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-08T01:21:56.690 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-08T01:21:56.690 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-08T01:21:56.691 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2023-08-08T01:21:56.712 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server 2023-08-08T01:21:56.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-08-08T01:21:56.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T01:21:56.812 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-08-08T01:21:57.002 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-08-08T01:21:57.003 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2023-08-08T01:21:57.058 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2023-08-08T01:21:57.058 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-08-08T01:21:57.113 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2023-08-08T01:21:57.113 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2023-08-08T01:21:57.113 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2023-08-08T01:21:57.113 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2023-08-08T01:21:57.114 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2023-08-08T01:21:57.114 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2023-08-08T01:21:57.114 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-08T01:21:57.114 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-08-08T01:21:57.115 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2023-08-08T01:21:57.115 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-08T01:21:57.115 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2023-08-08T01:21:57.115 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-08T01:21:57.115 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-08T01:21:57.115 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-08T01:21:57.116 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-08-08T01:21:57.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:21:57.228 INFO:teuthology.dispatcher.supervisor:Running job 7361188 2023-08-08T01:21:57.229 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-08-04_18:23:51-fs-wip-rishabh-2023aug3-testing-default-smithi/7361188 --name rishabh-2023-08-04_18:23:51-fs-wip-rishabh-2023aug3-testing-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} -- /home/teuthworker/archive/rishabh-2023-08-04_18:23:51-fs-wip-rishabh-2023aug3-testing-default-smithi/7361188/orig.config.yaml 2023-08-08T01:21:57.232 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-08-04_18:23:51-fs-wip-rishabh-2023aug3-testing-default-smithi/7361188 2023-08-08T01:21:57.233 INFO:teuthology.dispatcher.supervisor:Job PID: 20451 2023-08-08T01:21:57.233 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-08T01:23:57.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:25:57.420 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:27:57.577 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:29:57.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:31:57.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:33:58.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:35:58.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:37:58.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:39:58.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:41:58.695 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:43:58.861 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:45:59.029 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:47:59.174 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T01:47:59.235 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-08T01:47:59.393 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-08T01:47:59.393 INFO:teuthology.nuke:Checking targets against current locks 2023-08-08T01:47:59.442 DEBUG:teuthology.nuke:shortname: smithi039 2023-08-08T01:47:59.443 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-08T01:47:59.461 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-04_18:23:51-fs-wip-rishabh-2023aug3-testing-default-smithi/7361188', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-08 01:15:36.630503', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBDXLn40yVbXZz3d40yvhdV7QOC0wfQn9ZrVlhoOrQ0jdgtr1KlMiAzPP2CU3pDyrNOASJs++/Eq1VQU0bem64='} 2023-08-08T01:47:59.466 DEBUG:teuthology.nuke:shortname: smithi053 2023-08-08T01:47:59.466 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-08T01:47:59.485 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-04_18:23:51-fs-wip-rishabh-2023aug3-testing-default-smithi/7361188', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-08 01:15:36.628966', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC+o7AOuXmyYcVKVxk//3MxI5m2V+iIy0eOAiZWi4R/+0ka/tGKZavDTKK11FDkKOqb3haM5cAHk7jteVwj31h0='} 2023-08-08T01:47:59.515 INFO:teuthology.orchestra.console.smithi039:Power off 2023-08-08T01:47:59.516 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T01:47:59.537 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2023-08-08T01:47:59.537 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:47:59.554 INFO:teuthology.orchestra.console.smithi053:Power off 2023-08-08T01:47:59.555 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T01:47:59.563 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2023-08-08T01:47:59.574 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2023-08-08T01:47:59.574 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T01:47:59.598 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-08-08T01:47:59.664 INFO:teuthology.orchestra.console.smithi039:Power off completed 2023-08-08T01:47:59.699 INFO:teuthology.orchestra.console.smithi053:Power off completed 2023-08-08T01:47:59.810 INFO:teuthology.lock.ops:unlocked: smithi039.front.sepia.ceph.com 2023-08-08T01:47:59.842 INFO:teuthology.lock.ops:unlocked: smithi053.front.sepia.ceph.com