2023-11-13T06:00:51.408 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-13T06:00:51.409 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:00:51.545 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2207927b80>, signals=[15]) 2023-11-13T06:00:51.547 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging 2023-11-13T06:00:51.547 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server 2023-11-13T06:00:51.547 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2023-11-13T06:00:51.570 INFO:teuthology.lock.ops:Start node 'smithi124.front.sepia.ceph.com' reimaging 2023-11-13T06:00:51.570 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: reset os type and version on server 2023-11-13T06:00:51.571 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-11-13T06:00:51.572 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /host {"name": "smithi002"} 2023-11-13T06:00:51.595 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /host {"name": "smithi124"} 2023-11-13T06:00:51.703 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-13T06:00:51.706 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-13T06:00:51.749 DEBUG:teuthology.provision.fog.smithi002:Requesting image smithi_centos_9.stream (ID 10) 2023-11-13T06:00:51.749 DEBUG:teuthology.provision.fog.smithi002:FOG request: PUT /host/108 {"imageID": 10} 2023-11-13T06:00:51.752 DEBUG:teuthology.provision.fog.smithi124:Requesting image smithi_centos_9.stream (ID 10) 2023-11-13T06:00:51.752 DEBUG:teuthology.provision.fog.smithi124:FOG request: PUT /host/229 {"imageID": 10} 2023-11-13T06:00:51.793 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of centos 9.stream 2023-11-13T06:00:51.794 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:00:51.797 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 9.stream 2023-11-13T06:00:51.798 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:00:51.836 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /tasktype {"name": "deploy"} 2023-11-13T06:00:51.840 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /tasktype {"name": "deploy"} 2023-11-13T06:00:51.870 DEBUG:teuthology.provision.fog.smithi002:FOG request: POST /host/108/task {"taskTypeID": 1} 2023-11-13T06:00:51.875 DEBUG:teuthology.provision.fog.smithi124:FOG request: POST /host/229/task {"taskTypeID": 1} 2023-11-13T06:00:51.970 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:00:51.983 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:00:52.045 INFO:teuthology.orchestra.console.smithi002:Power off 2023-11-13T06:00:52.046 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T06:00:52.058 INFO:teuthology.orchestra.console.smithi124:Power off 2023-11-13T06:00:52.058 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T06:00:52.070 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2023-11-13T06:00:52.071 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:00:52.078 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2023-11-13T06:00:52.079 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:00:52.092 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2023-11-13T06:00:52.099 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-11-13T06:00:56.098 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:00:56.107 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:00:56.210 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2023-11-13T06:00:56.218 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-11-13T06:01:00.213 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:01:00.221 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:01:00.323 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2023-11-13T06:01:00.330 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-11-13T06:01:00.425 INFO:teuthology.orchestra.console.smithi002:Power off completed 2023-11-13T06:01:00.431 INFO:teuthology.orchestra.console.smithi124:Power off completed 2023-11-13T06:01:00.526 INFO:teuthology.orchestra.console.smithi002:Power on 2023-11-13T06:01:00.526 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-13T06:01:00.532 INFO:teuthology.orchestra.console.smithi124:Power on 2023-11-13T06:01:00.533 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-13T06:01:00.546 DEBUG:teuthology.orchestra.console.smithi002:power on output: Chassis Power Control: Up/On 2023-11-13T06:01:00.589 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:01:00.597 DEBUG:teuthology.orchestra.console.smithi124:power on output: Chassis Power Control: Up/On 2023-11-13T06:01:00.597 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:01:00.610 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2023-11-13T06:01:00.619 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-11-13T06:01:04.615 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:01:04.623 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:01:04.725 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2023-11-13T06:01:04.731 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-11-13T06:01:08.731 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:01:08.737 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:01:08.839 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2023-11-13T06:01:08.844 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-11-13T06:01:08.941 INFO:teuthology.orchestra.console.smithi002:Power on completed 2023-11-13T06:01:08.945 INFO:teuthology.orchestra.console.smithi124:Power on completed 2023-11-13T06:01:09.042 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2023-11-13T06:01:09.042 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:01:09.045 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish 2023-11-13T06:01:09.046 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:01:24.127 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:01:24.175 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:01:39.196 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:01:39.226 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:01:54.284 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:01:54.286 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:02:09.367 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:02:09.368 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:02:24.431 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:02:24.434 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:02:39.506 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:02:39.509 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:02:54.581 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:02:54.583 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:03:09.653 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:03:09.658 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:03:24.726 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:03:24.727 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:03:39.799 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-13T06:03:39.801 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-11-13T06:03:39.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:03:39.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:03:58.203 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-11-13T06:03:58.204 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-13T06:04:04.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:04:04.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:04:07.291 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-11-13T06:04:07.291 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-13T06:04:13.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:04:13.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:04:16.346 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-11-13T06:04:16.378 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-13T06:04:22.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:04:22.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:04:25.434 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-11-13T06:04:25.435 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-13T06:04:31.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:04:31.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:04:34.523 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-13T06:04:40.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:05:31.497 WARNING:teuthology.provision.fog.smithi002:timed out 2023-11-13T06:05:37.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:05:37.786 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-13T06:05:37.787 DEBUG:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-13T06:05:38.114 INFO:teuthology.provision.fog.smithi002:Node is ready 2023-11-13T06:05:38.115 DEBUG:teuthology.orchestra.run.smithi002:> hostname 2023-11-13T06:05:38.130 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2023-11-13T06:05:38.131 DEBUG:teuthology.orchestra.run.smithi002:> grep smithi002.front.sepia.ceph.com /etc/hosts 2023-11-13T06:05:38.186 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2023-11-13T06:05:38.187 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2023-11-13T06:05:38.299 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hosts 2023-11-13T06:05:38.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/172.21.15.2/172.21.15.2/g' /etc/hosts 2023-11-13T06:05:38.402 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hostname 2023-11-13T06:05:38.473 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2023-11-13T06:05:38.543 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2023-11-13T06:05:38.598 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2023-11-13T06:05:38.598 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2023-11-13T06:05:38.598 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2023-11-13T06:05:38.599 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2023-11-13T06:05:38.599 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2023-11-13T06:05:38.599 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2023-11-13T06:05:38.600 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-13T06:05:38.600 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2023-11-13T06:05:38.600 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2023-11-13T06:05:38.601 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-13T06:05:38.601 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2023-11-13T06:05:38.601 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T06:05:38.602 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-13T06:05:38.602 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T06:05:38.603 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2023-11-13T06:05:38.605 DEBUG:teuthology.parallel:result is None 2023-11-13T06:05:40.535 WARNING:teuthology.provision.fog.smithi124:timed out 2023-11-13T06:05:46.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:05:46.765 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-13T06:05:46.766 DEBUG:teuthology.orchestra.run.smithi124:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-13T06:05:47.087 INFO:teuthology.provision.fog.smithi124:Node is ready 2023-11-13T06:05:47.087 DEBUG:teuthology.orchestra.run.smithi124:> hostname 2023-11-13T06:05:47.102 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com 2023-11-13T06:05:47.103 DEBUG:teuthology.orchestra.run.smithi124:> grep smithi124.front.sepia.ceph.com /etc/hosts 2023-11-13T06:05:47.158 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124 2023-11-13T06:05:47.158 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124 2023-11-13T06:05:47.269 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi124.front.sepia.ceph.com/smithi124/g' /etc/hosts 2023-11-13T06:05:47.298 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/172.21.15.124/172.21.15.124/g' /etc/hosts 2023-11-13T06:05:47.366 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi124.front.sepia.ceph.com/smithi124/g' /etc/hostname 2023-11-13T06:05:47.433 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124 2023-11-13T06:05:47.500 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2023-11-13T06:05:47.555 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Stream" 2023-11-13T06:05:47.555 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="9" 2023-11-13T06:05:47.556 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2023-11-13T06:05:47.556 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2023-11-13T06:05:47.556 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="9" 2023-11-13T06:05:47.556 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el9" 2023-11-13T06:05:47.557 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-13T06:05:47.557 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2023-11-13T06:05:47.557 INFO:teuthology.orchestra.run.smithi124.stdout:LOGO="fedora-logo-icon" 2023-11-13T06:05:47.558 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-13T06:05:47.558 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://centos.org/" 2023-11-13T06:05:47.558 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T06:05:47.558 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-13T06:05:47.559 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T06:05:47.559 INFO:teuthology.provision.fog.smithi124:Deploy complete! 2023-11-13T06:05:47.560 DEBUG:teuthology.parallel:result is None 2023-11-13T06:05:47.747 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2023-11-13T06:05:47.748 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-13T06:05:47.778 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com 2023-11-13T06:05:47.778 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-13T06:05:47.804 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2023-11-13T06:05:47.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-11-13T06:05:47.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:05:47.906 DEBUG:teuthology.orchestra.run.smithi002:> true 2023-11-13T06:05:47.979 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-11-13T06:05:47.980 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2023-11-13T06:05:48.040 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2023-11-13T06:05:48.042 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2023-11-13T06:05:48.096 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2023-11-13T06:05:48.097 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2023-11-13T06:05:48.097 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2023-11-13T06:05:48.097 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2023-11-13T06:05:48.098 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2023-11-13T06:05:48.098 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2023-11-13T06:05:48.098 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-13T06:05:48.099 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2023-11-13T06:05:48.099 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2023-11-13T06:05:48.099 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-13T06:05:48.099 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2023-11-13T06:05:48.100 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T06:05:48.100 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-13T06:05:48.100 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T06:05:48.102 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2023-11-13T06:05:48.129 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server 2023-11-13T06:05:48.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-11-13T06:05:48.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T06:05:48.212 DEBUG:teuthology.orchestra.run.smithi124:> true 2023-11-13T06:05:48.278 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-11-13T06:05:48.278 DEBUG:teuthology.orchestra.run.smithi124:> uname -m 2023-11-13T06:05:48.332 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2023-11-13T06:05:48.333 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2023-11-13T06:05:48.386 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Stream" 2023-11-13T06:05:48.387 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="9" 2023-11-13T06:05:48.387 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2023-11-13T06:05:48.387 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2023-11-13T06:05:48.387 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="9" 2023-11-13T06:05:48.388 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el9" 2023-11-13T06:05:48.388 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-13T06:05:48.388 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2023-11-13T06:05:48.388 INFO:teuthology.orchestra.run.smithi124.stdout:LOGO="fedora-logo-icon" 2023-11-13T06:05:48.389 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-13T06:05:48.389 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://centos.org/" 2023-11-13T06:05:48.389 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T06:05:48.389 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-13T06:05:48.390 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T06:05:48.391 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-11-13T06:05:48.420 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:05:48.528 INFO:teuthology.dispatcher.supervisor:Running job 7456097 2023-11-13T06:05:48.529 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_nmordech@teuthology --archive /home/teuthworker/archive/nmordech-2023-11-13_05:59:55-rados:thrash-dedup-main-distro-default-smithi/7456097 --name nmordech-2023-11-13_05:59:55-rados:thrash-dedup-main-distro-default-smithi --description rados:thrash-dedup/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/dedup-io-mixed} -- /home/teuthworker/archive/nmordech-2023-11-13_05:59:55-rados:thrash-dedup-main-distro-default-smithi/7456097/orig.config.yaml 2023-11-13T06:05:48.533 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nmordech-2023-11-13_05:59:55-rados:thrash-dedup-main-distro-default-smithi/7456097 2023-11-13T06:05:48.534 INFO:teuthology.dispatcher.supervisor:Job PID: 2660975 2023-11-13T06:05:48.534 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-13T06:07:48.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:09:48.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:11:49.124 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:13:49.343 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:15:49.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:17:49.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:19:49.760 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:21:49.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:23:50.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:25:50.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:27:50.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:29:51.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:31:51.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:33:51.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:35:51.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:37:51.658 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T06:37:51.764 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-13T06:37:51.871 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-13T06:37:51.872 INFO:teuthology.nuke:Checking targets against current locks 2023-11-13T06:37:51.925 DEBUG:teuthology.nuke:shortname: smithi002 2023-11-13T06:37:51.925 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T06:37:51.948 DEBUG:teuthology.nuke:shortname: smithi124 2023-11-13T06:37:51.948 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T06:37:51.952 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/nmordech-2023-11-13_05:59:55-rados:thrash-dedup-main-distro-default-smithi/7456097', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 06:00:50.677828', 'locked_by': 'scheduled_nmordech@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNe97lKKYl1SOaJya67ka2G90I1zueZejCeRFqQVZllY8A6tl6C9oiD8zpY8AN89RKxn8qUX8Xmd0NxLl8yNf24='} 2023-11-13T06:37:51.974 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/nmordech-2023-11-13_05:59:55-rados:thrash-dedup-main-distro-default-smithi/7456097', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 06:00:50.676156', 'locked_by': 'scheduled_nmordech@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgcJLEzPi51Mq6VGNxbgiJdGfg9MQQ/s6dro4swLkj7fuiTPi4bVEF+Opk6zSVQLn1mGoBKzImOjQxC8+DO4tI='} 2023-11-13T06:37:52.006 INFO:teuthology.orchestra.console.smithi002:Power off 2023-11-13T06:37:52.007 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T06:37:52.045 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2023-11-13T06:37:52.046 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:37:52.063 INFO:teuthology.orchestra.console.smithi124:Power off 2023-11-13T06:37:52.063 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T06:37:52.072 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2023-11-13T06:37:52.085 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2023-11-13T06:37:52.085 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T06:37:52.106 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-11-13T06:37:52.173 INFO:teuthology.orchestra.console.smithi002:Power off completed 2023-11-13T06:37:52.208 INFO:teuthology.orchestra.console.smithi124:Power off completed 2023-11-13T06:37:52.339 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com 2023-11-13T06:37:52.355 INFO:teuthology.lock.ops:unlocked: smithi124.front.sepia.ceph.com