2023-11-14T22:06:37.237 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-14T22:06:37.238 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:06:37.399 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f72586b9ac0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f72591a4670>, signals=[15])
2023-11-14T22:06:37.401 INFO:teuthology.lock.ops:Start node 'smithi091.front.sepia.ceph.com' reimaging
2023-11-14T22:06:37.401 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: reset os type and version on server
2023-11-14T22:06:37.401 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server
2023-11-14T22:06:37.425 INFO:teuthology.lock.ops:Start node 'smithi157.front.sepia.ceph.com' reimaging
2023-11-14T22:06:37.426 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: reset os type and version on server
2023-11-14T22:06:37.426 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server
2023-11-14T22:06:37.428 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /host {"name": "smithi091"}
2023-11-14T22:06:37.451 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /host {"name": "smithi157"}
2023-11-14T22:06:37.512 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-11-14T22:06:37.518 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-11-14T22:06:37.554 DEBUG:teuthology.provision.fog.smithi091:Requesting image smithi_rhel_8.4 (ID 7)
2023-11-14T22:06:37.555 DEBUG:teuthology.provision.fog.smithi091:FOG request: PUT /host/196 {"imageID": 7}
2023-11-14T22:06:37.559 DEBUG:teuthology.provision.fog.smithi157:Requesting image smithi_rhel_8.4 (ID 7)
2023-11-14T22:06:37.559 DEBUG:teuthology.provision.fog.smithi157:FOG request: PUT /host/262 {"imageID": 7}
2023-11-14T22:06:37.593 INFO:teuthology.provision.fog.smithi091:Scheduling deploy of rhel 8.4
2023-11-14T22:06:37.593 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:06:37.607 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of rhel 8.4
2023-11-14T22:06:37.608 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:06:37.642 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /tasktype {"name": "deploy"}
2023-11-14T22:06:37.645 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /tasktype {"name": "deploy"}
2023-11-14T22:06:37.679 DEBUG:teuthology.provision.fog.smithi157:FOG request: POST /host/262/task {"taskTypeID": 1}
2023-11-14T22:06:37.683 DEBUG:teuthology.provision.fog.smithi091:FOG request: POST /host/196/task {"taskTypeID": 1}
2023-11-14T22:06:37.731 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:06:37.735 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:06:37.822 INFO:teuthology.orchestra.console.smithi157:Power off
2023-11-14T22:06:37.823 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-14T22:06:37.848 INFO:teuthology.orchestra.console.smithi091:Power off
2023-11-14T22:06:37.848 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-14T22:06:37.856 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off
2023-11-14T22:06:37.857 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:37.877 DEBUG:teuthology.orchestra.console.smithi091:power off output: Chassis Power Control: Down/Off
2023-11-14T22:06:37.877 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:37.886 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off
2023-11-14T22:06:37.900 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on
2023-11-14T22:06:37.988 INFO:teuthology.orchestra.console.smithi157:Power off completed
2023-11-14T22:06:38.089 INFO:teuthology.orchestra.console.smithi157:Power on
2023-11-14T22:06:38.089 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-14T22:06:38.114 DEBUG:teuthology.orchestra.console.smithi157:power on output: Chassis Power Control: Up/On
2023-11-14T22:06:38.114 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:38.139 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off
2023-11-14T22:06:41.902 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:42.014 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on
2023-11-14T22:06:42.141 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:42.252 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on
2023-11-14T22:06:42.353 INFO:teuthology.orchestra.console.smithi157:Power on completed
2023-11-14T22:06:42.454 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish
2023-11-14T22:06:42.454 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:06:46.018 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:46.127 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off
2023-11-14T22:06:46.228 INFO:teuthology.orchestra.console.smithi091:Power off completed
2023-11-14T22:06:46.330 INFO:teuthology.orchestra.console.smithi091:Power on
2023-11-14T22:06:46.330 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-14T22:06:46.353 DEBUG:teuthology.orchestra.console.smithi091:power on output: Chassis Power Control: Up/On
2023-11-14T22:06:46.354 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:46.379 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off
2023-11-14T22:06:50.383 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:50.494 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off
2023-11-14T22:06:54.498 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:06:54.715 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on
2023-11-14T22:06:54.816 INFO:teuthology.orchestra.console.smithi091:Power on completed
2023-11-14T22:06:54.917 INFO:teuthology.provision.fog.smithi091:Waiting for deploy to finish
2023-11-14T22:06:54.917 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:06:57.516 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:07:09.985 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:07:12.566 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:07:25.053 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:07:27.622 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:07:40.130 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:07:42.686 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:07:55.218 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:07:57.759 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:08:10.291 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:08:12.830 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:08:25.365 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:08:27.898 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:08:40.438 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:08:42.970 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:08:55.510 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:08:58.034 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:09:10.586 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:09:13.101 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:09:25.670 DEBUG:teuthology.provision.fog.smithi091:FOG request: GET /task/active None
2023-11-14T22:09:25.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:09:25.831 ERROR:teuthology.orchestra.connection:Error authenticating with smithi091.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-14T22:09:28.173 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:09:43.246 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:09:58.334 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None
2023-11-14T22:09:58.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:01.466 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:10:07.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:10.522 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:10:16.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:19.578 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:10:25.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:28.634 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:10:29.835 WARNING:teuthology.provision.fog.smithi091:timed out
2023-11-14T22:10:34.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:35.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:37.690 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:10:38.906 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-14T22:10:43.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:44.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:46.750 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:10:47.962 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-14T22:10:52.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:53.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:10:55.802 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:10:57.018 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-14T22:11:01.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:11:03.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:11:04.858 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:11:06.078 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-14T22:11:10.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:11:12.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:11:13.136 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-14T22:11:13.137 DEBUG:teuthology.orchestra.run.smithi091:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-14T22:11:28.737 INFO:teuthology.provision.fog.smithi091:Node is ready
2023-11-14T22:11:28.737 DEBUG:teuthology.orchestra.run.smithi091:> hostname
2023-11-14T22:11:28.793 INFO:teuthology.orchestra.run.smithi091.stdout:smithi091.front.sepia.ceph.com
2023-11-14T22:11:28.793 DEBUG:teuthology.orchestra.run.smithi091:> grep smithi091.front.sepia.ceph.com /etc/hosts
2023-11-14T22:11:28.847 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.15.91 smithi091.front.sepia.ceph.com smithi091
2023-11-14T22:11:28.848 DEBUG:teuthology.orchestra.run.smithi091:> sudo hostname smithi091
2023-11-14T22:11:28.960 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/smithi091.front.sepia.ceph.com/smithi091/g' /etc/hosts
2023-11-14T22:11:28.985 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/172.21.15.91/172.21.15.91/g' /etc/hosts
2023-11-14T22:11:29.049 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/smithi091.front.sepia.ceph.com/smithi091/g' /etc/hostname
2023-11-14T22:11:29.113 DEBUG:teuthology.orchestra.run.smithi091:> sudo hostname smithi091
2023-11-14T22:11:29.177 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release
2023-11-14T22:11:29.232 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="Red Hat Enterprise Linux"
2023-11-14T22:11:29.232 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="8.4 (Ootpa)"
2023-11-14T22:11:29.232 INFO:teuthology.orchestra.run.smithi091.stdout:ID="rhel"
2023-11-14T22:11:29.232 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="fedora"
2023-11-14T22:11:29.233 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="8.4"
2023-11-14T22:11:29.233 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el8"
2023-11-14T22:11:29.233 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-14T22:11:29.233 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31"
2023-11-14T22:11:29.234 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-14T22:11:29.234 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.redhat.com/"
2023-11-14T22:11:29.234 INFO:teuthology.orchestra.run.smithi091.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-14T22:11:29.234 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-14T22:11:29.235 INFO:teuthology.orchestra.run.smithi091.stdout:
2023-11-14T22:11:29.235 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-14T22:11:29.235 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-14T22:11:29.235 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-14T22:11:29.236 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-14T22:11:29.236 INFO:teuthology.provision.fog.smithi091:Deploy complete!
2023-11-14T22:11:29.238 DEBUG:teuthology.parallel:result is None
2023-11-14T22:11:29.274 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:11:35.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:11:38.334 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:11:44.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:11:47.386 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157
2023-11-14T22:11:53.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:11:53.694 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-14T22:11:53.695 DEBUG:teuthology.orchestra.run.smithi157:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-14T22:12:14.047 INFO:teuthology.provision.fog.smithi157:Node is ready
2023-11-14T22:12:14.048 DEBUG:teuthology.orchestra.run.smithi157:> hostname
2023-11-14T22:12:14.064 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com
2023-11-14T22:12:14.065 DEBUG:teuthology.orchestra.run.smithi157:> grep smithi157.front.sepia.ceph.com /etc/hosts
2023-11-14T22:12:14.119 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157
2023-11-14T22:12:14.120 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157
2023-11-14T22:12:14.267 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hosts
2023-11-14T22:12:14.291 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/172.21.15.157/172.21.15.157/g' /etc/hosts
2023-11-14T22:12:14.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hostname
2023-11-14T22:12:14.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157
2023-11-14T22:12:14.495 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release
2023-11-14T22:12:14.553 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux"
2023-11-14T22:12:14.553 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.4 (Ootpa)"
2023-11-14T22:12:14.554 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel"
2023-11-14T22:12:14.554 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora"
2023-11-14T22:12:14.554 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.4"
2023-11-14T22:12:14.554 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8"
2023-11-14T22:12:14.554 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-14T22:12:14.555 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31"
2023-11-14T22:12:14.555 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-14T22:12:14.555 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/"
2023-11-14T22:12:14.555 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-14T22:12:14.556 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-14T22:12:14.556 INFO:teuthology.orchestra.run.smithi157.stdout:
2023-11-14T22:12:14.556 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-14T22:12:14.556 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-14T22:12:14.556 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-14T22:12:14.556 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-14T22:12:14.557 INFO:teuthology.provision.fog.smithi157:Deploy complete!
2023-11-14T22:12:14.557 DEBUG:teuthology.parallel:result is None
2023-11-14T22:12:14.738 INFO:teuthology.lock.ops:Checking smithi091.front.sepia.ceph.com
2023-11-14T22:12:14.738 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-14T22:12:14.764 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com
2023-11-14T22:12:14.765 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-14T22:12:14.790 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: set os type and version on server
2023-11-14T22:12:14.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com'
2023-11-14T22:12:14.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:12:14.858 DEBUG:teuthology.orchestra.run.smithi091:> true
2023-11-14T22:12:14.933 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi091.front.sepia.ceph.com'
2023-11-14T22:12:14.934 DEBUG:teuthology.orchestra.run.smithi091:> uname -m
2023-11-14T22:12:14.988 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64
2023-11-14T22:12:14.988 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release
2023-11-14T22:12:15.043 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="Red Hat Enterprise Linux"
2023-11-14T22:12:15.043 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="8.4 (Ootpa)"
2023-11-14T22:12:15.043 INFO:teuthology.orchestra.run.smithi091.stdout:ID="rhel"
2023-11-14T22:12:15.044 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="fedora"
2023-11-14T22:12:15.044 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="8.4"
2023-11-14T22:12:15.044 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el8"
2023-11-14T22:12:15.044 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-14T22:12:15.044 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31"
2023-11-14T22:12:15.044 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-14T22:12:15.045 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.redhat.com/"
2023-11-14T22:12:15.045 INFO:teuthology.orchestra.run.smithi091.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-14T22:12:15.045 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-14T22:12:15.045 INFO:teuthology.orchestra.run.smithi091.stdout:
2023-11-14T22:12:15.045 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-14T22:12:15.046 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-14T22:12:15.046 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-14T22:12:15.046 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-14T22:12:15.047 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server
2023-11-14T22:12:15.073 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server
2023-11-14T22:12:15.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com'
2023-11-14T22:12:15.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T22:12:15.145 DEBUG:teuthology.orchestra.run.smithi157:> true
2023-11-14T22:12:15.215 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com'
2023-11-14T22:12:15.216 DEBUG:teuthology.orchestra.run.smithi157:> uname -m
2023-11-14T22:12:15.273 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64
2023-11-14T22:12:15.274 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release
2023-11-14T22:12:15.331 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux"
2023-11-14T22:12:15.331 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.4 (Ootpa)"
2023-11-14T22:12:15.331 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel"
2023-11-14T22:12:15.332 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora"
2023-11-14T22:12:15.332 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.4"
2023-11-14T22:12:15.332 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8"
2023-11-14T22:12:15.332 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-14T22:12:15.332 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31"
2023-11-14T22:12:15.333 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-14T22:12:15.333 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/"
2023-11-14T22:12:15.333 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-14T22:12:15.334 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-14T22:12:15.334 INFO:teuthology.orchestra.run.smithi157.stdout:
2023-11-14T22:12:15.334 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-14T22:12:15.334 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-14T22:12:15.335 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-14T22:12:15.335 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-14T22:12:15.335 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server
2023-11-14T22:12:15.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:12:15.441 INFO:teuthology.dispatcher.supervisor:Running job 7457818
2023-11-14T22:12:15.442 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-14_15:40:04-rados-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457818 --name yuriw-2023-11-14_15:40:04-rados-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi --description rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} -- /home/teuthworker/archive/yuriw-2023-11-14_15:40:04-rados-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457818/orig.config.yaml
2023-11-14T22:12:15.445 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-14_15:40:04-rados-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457818
2023-11-14T22:12:15.446 INFO:teuthology.dispatcher.supervisor:Job PID: 2500109
2023-11-14T22:12:15.446 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-14T22:14:15.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:16:15.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:18:15.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:20:15.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:22:15.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:24:15.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:26:15.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:28:15.992 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:30:16.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:32:16.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:34:16.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:36:16.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:38:16.358 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T22:38:16.432 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-11-14T22:38:16.588 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-14T22:38:16.589 INFO:teuthology.nuke:Checking targets against current locks
2023-11-14T22:38:16.641 DEBUG:teuthology.nuke:shortname: smithi091
2023-11-14T22:38:16.641 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-14T22:38:16.663 DEBUG:teuthology.nuke:shortname: smithi157
2023-11-14T22:38:16.664 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-14T22:38:16.668 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi091.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-14_15:40:04-rados-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457818', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 22:06:36.310580', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWWmQcg7bjfZlw+HVJptTcHcsOF8uZ7B8Qu1b62/zXstUyQqgKxd0bloXKEj/KE2nl4B3pJWdDQhDBQlp5O0M4='}
2023-11-14T22:38:16.690 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-14_15:40:04-rados-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457818', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 22:06:36.312561', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMn8FZfjuSLvQQHc8vlJqITOVqLdIhQDSoM8d0mM1rImcL1b36vD9iZOlFytbSao5eXXA60OLk4GTEGVsPwv9O8='}
2023-11-14T22:38:16.724 INFO:teuthology.orchestra.console.smithi091:Power off
2023-11-14T22:38:16.725 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-14T22:38:16.753 DEBUG:teuthology.orchestra.console.smithi091:power off output: Chassis Power Control: Down/Off
2023-11-14T22:38:16.753 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:38:16.774 INFO:teuthology.orchestra.console.smithi157:Power off
2023-11-14T22:38:16.774 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-14T22:38:16.783 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off
2023-11-14T22:38:16.799 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off
2023-11-14T22:38:16.799 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T22:38:16.824 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off
2023-11-14T22:38:16.885 INFO:teuthology.orchestra.console.smithi091:Power off completed
2023-11-14T22:38:16.925 INFO:teuthology.orchestra.console.smithi157:Power off completed
2023-11-14T22:38:17.036 INFO:teuthology.lock.ops:unlocked: smithi091.front.sepia.ceph.com
2023-11-14T22:38:17.070 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com