2023-08-08T20:47:29.324 INFO:root:teuthology version: 0.0.1.dev181+g7fda959
2023-08-08T20:47:29.326 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T20:47:29.481 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7efd139387f0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7efd13210ee0>, signals=[15])
2023-08-08T20:47:29.482 INFO:teuthology.lock.ops:Start node 'smithi090.front.sepia.ceph.com' reimaging
2023-08-08T20:47:29.483 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: reset os type and version on server
2023-08-08T20:47:29.483 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2023-08-08T20:47:29.506 INFO:teuthology.lock.ops:Start node 'smithi137.front.sepia.ceph.com' reimaging
2023-08-08T20:47:29.506 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: reset os type and version on server
2023-08-08T20:47:29.507 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server
2023-08-08T20:47:29.509 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /host {"name": "smithi090"}
2023-08-08T20:47:29.530 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /host {"name": "smithi137"}
2023-08-08T20:47:29.581 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-08-08T20:47:29.603 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-08-08T20:47:29.621 DEBUG:teuthology.provision.fog.smithi090:Requesting image smithi_centos_9.stream (ID 10)
2023-08-08T20:47:29.622 DEBUG:teuthology.provision.fog.smithi090:FOG request: PUT /host/195 {"imageID": 10}
2023-08-08T20:47:29.646 DEBUG:teuthology.provision.fog.smithi137:Requesting image smithi_centos_9.stream (ID 10)
2023-08-08T20:47:29.647 DEBUG:teuthology.provision.fog.smithi137:FOG request: PUT /host/242 {"imageID": 10}
2023-08-08T20:47:29.660 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of centos 9.stream
2023-08-08T20:47:29.660 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:47:29.693 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of centos 9.stream
2023-08-08T20:47:29.694 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:47:29.733 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /tasktype {"name": "deploy"}
2023-08-08T20:47:29.764 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /tasktype {"name": "deploy"}
2023-08-08T20:47:29.771 DEBUG:teuthology.provision.fog.smithi090:FOG request: POST /host/195/task {"taskTypeID": 1}
2023-08-08T20:47:29.811 DEBUG:teuthology.provision.fog.smithi137:FOG request: POST /host/242/task {"taskTypeID": 1}
2023-08-08T20:47:29.834 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:47:29.865 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:47:29.953 INFO:teuthology.orchestra.console.smithi090:Power off
2023-08-08T20:47:29.954 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-08T20:47:29.979 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off
2023-08-08T20:47:29.980 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:29.997 INFO:teuthology.orchestra.console.smithi137:Power off
2023-08-08T20:47:29.998 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-08T20:47:30.007 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on
2023-08-08T20:47:30.019 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off
2023-08-08T20:47:30.020 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:30.044 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on
2023-08-08T20:47:34.012 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:34.046 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:34.126 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on
2023-08-08T20:47:34.156 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on
2023-08-08T20:47:38.128 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:38.157 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:38.240 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on
2023-08-08T20:47:38.271 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on
2023-08-08T20:47:42.245 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:42.271 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:42.356 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off
2023-08-08T20:47:42.383 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off
2023-08-08T20:47:42.458 INFO:teuthology.orchestra.console.smithi090:Power off completed
2023-08-08T20:47:42.484 INFO:teuthology.orchestra.console.smithi137:Power off completed
2023-08-08T20:47:42.559 INFO:teuthology.orchestra.console.smithi090:Power on
2023-08-08T20:47:42.559 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-08T20:47:42.585 INFO:teuthology.orchestra.console.smithi137:Power on
2023-08-08T20:47:42.586 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-08T20:47:42.609 DEBUG:teuthology.orchestra.console.smithi137:power on output: Chassis Power Control: Up/On
2023-08-08T20:47:42.610 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:42.621 DEBUG:teuthology.orchestra.console.smithi090:power on output: Chassis Power Control: Up/On
2023-08-08T20:47:42.622 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:42.634 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off
2023-08-08T20:47:42.646 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off
2023-08-08T20:47:46.640 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:46.648 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:46.751 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off
2023-08-08T20:47:46.759 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on
2023-08-08T20:47:46.860 INFO:teuthology.orchestra.console.smithi090:Power on completed
2023-08-08T20:47:46.961 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish
2023-08-08T20:47:46.961 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:47:50.756 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T20:47:50.866 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on
2023-08-08T20:47:50.967 INFO:teuthology.orchestra.console.smithi137:Power on completed
2023-08-08T20:47:51.069 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish
2023-08-08T20:47:51.070 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:48:02.045 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:48:06.152 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:48:17.164 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:48:21.244 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:48:32.258 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:48:36.328 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:48:47.356 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:48:51.416 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:49:02.448 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:49:06.501 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:49:17.528 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:49:21.572 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:49:32.620 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:49:36.664 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:49:47.716 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:49:51.744 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:50:02.808 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:50:06.824 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:50:17.900 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:50:21.908 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None
2023-08-08T20:50:21.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:50:25.076 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-08-08T20:50:31.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:50:32.978 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:50:34.160 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-08-08T20:50:40.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:50:43.220 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-08-08T20:50:48.060 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-08-08T20:50:48.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:50:49.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:50:51.184 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-08-08T20:50:52.276 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-08-08T20:50:57.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:50:58.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:00.240 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-08-08T20:51:01.360 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-08-08T20:51:06.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:07.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:09.296 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-08-08T20:51:10.416 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-08-08T20:51:15.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:16.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:18.352 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-08-08T20:51:19.472 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-08-08T20:51:24.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:25.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:27.408 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-08-08T20:51:33.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:36.465 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-08-08T20:51:42.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:45.552 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-08-08T20:51:51.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:51:54.608 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-08-08T20:52:00.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:52:25.500 WARNING:teuthology.provision.fog.smithi137:timed out
2023-08-08T20:52:31.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:52:31.783 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-08T20:52:31.784 DEBUG:teuthology.orchestra.run.smithi137:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-08T20:52:32.116 INFO:teuthology.provision.fog.smithi137:Node is ready
2023-08-08T20:52:32.116 DEBUG:teuthology.orchestra.run.smithi137:> hostname
2023-08-08T20:52:32.132 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com
2023-08-08T20:52:32.132 DEBUG:teuthology.orchestra.run.smithi137:> grep smithi137.front.sepia.ceph.com /etc/hosts
2023-08-08T20:52:32.188 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137
2023-08-08T20:52:32.189 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137
2023-08-08T20:52:32.311 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hosts
2023-08-08T20:52:32.339 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/172.21.15.137/172.21.15.137/g' /etc/hosts
2023-08-08T20:52:32.406 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hostname
2023-08-08T20:52:32.473 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137
2023-08-08T20:52:32.540 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release
2023-08-08T20:52:32.594 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream"
2023-08-08T20:52:32.595 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9"
2023-08-08T20:52:32.595 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos"
2023-08-08T20:52:32.595 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora"
2023-08-08T20:52:32.595 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9"
2023-08-08T20:52:32.595 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9"
2023-08-08T20:52:32.595 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9"
2023-08-08T20:52:32.595 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31"
2023-08-08T20:52:32.596 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon"
2023-08-08T20:52:32.596 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-08-08T20:52:32.596 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/"
2023-08-08T20:52:32.596 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-08T20:52:32.596 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-08-08T20:52:32.596 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-08-08T20:52:32.597 INFO:teuthology.provision.fog.smithi137:Deploy complete!
2023-08-08T20:52:32.598 DEBUG:teuthology.parallel:result is None
2023-08-08T20:53:00.640 WARNING:teuthology.provision.fog.smithi090:timed out
2023-08-08T20:53:06.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:53:06.881 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-08T20:53:06.881 DEBUG:teuthology.orchestra.run.smithi090:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-08T20:53:07.268 INFO:teuthology.provision.fog.smithi090:Node is ready
2023-08-08T20:53:07.269 DEBUG:teuthology.orchestra.run.smithi090:> hostname
2023-08-08T20:53:07.285 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090.front.sepia.ceph.com
2023-08-08T20:53:07.285 DEBUG:teuthology.orchestra.run.smithi090:> grep smithi090.front.sepia.ceph.com /etc/hosts
2023-08-08T20:53:07.341 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90 smithi090.front.sepia.ceph.com smithi090
2023-08-08T20:53:07.342 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090
2023-08-08T20:53:07.463 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi090.front.sepia.ceph.com/smithi090/g' /etc/hosts
2023-08-08T20:53:07.492 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/172.21.15.90/172.21.15.90/g' /etc/hosts
2023-08-08T20:53:07.560 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi090.front.sepia.ceph.com/smithi090/g' /etc/hostname
2023-08-08T20:53:07.628 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090
2023-08-08T20:53:07.695 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release
2023-08-08T20:53:07.750 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream"
2023-08-08T20:53:07.750 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9"
2023-08-08T20:53:07.750 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos"
2023-08-08T20:53:07.750 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora"
2023-08-08T20:53:07.750 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9"
2023-08-08T20:53:07.751 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9"
2023-08-08T20:53:07.751 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9"
2023-08-08T20:53:07.751 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2023-08-08T20:53:07.751 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon"
2023-08-08T20:53:07.751 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-08-08T20:53:07.751 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/"
2023-08-08T20:53:07.752 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-08T20:53:07.752 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-08-08T20:53:07.752 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-08-08T20:53:07.752 INFO:teuthology.provision.fog.smithi090:Deploy complete!
2023-08-08T20:53:07.753 DEBUG:teuthology.parallel:result is None
2023-08-08T20:53:07.897 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com
2023-08-08T20:53:07.898 INFO:teuthology.lock.ops:New key found. Updating...
2023-08-08T20:53:07.916 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com
2023-08-08T20:53:07.916 INFO:teuthology.lock.ops:New key found. Updating...
2023-08-08T20:53:07.933 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server
2023-08-08T20:53:07.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2023-08-08T20:53:07.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:53:08.017 DEBUG:teuthology.orchestra.run.smithi090:> true
2023-08-08T20:53:08.041 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com'
2023-08-08T20:53:08.042 DEBUG:teuthology.orchestra.run.smithi090:> uname -m
2023-08-08T20:53:08.095 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64
2023-08-08T20:53:08.095 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release
2023-08-08T20:53:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream"
2023-08-08T20:53:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9"
2023-08-08T20:53:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos"
2023-08-08T20:53:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora"
2023-08-08T20:53:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9"
2023-08-08T20:53:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9"
2023-08-08T20:53:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9"
2023-08-08T20:53:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2023-08-08T20:53:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon"
2023-08-08T20:53:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-08-08T20:53:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/"
2023-08-08T20:53:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-08T20:53:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-08-08T20:53:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-08-08T20:53:08.151 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2023-08-08T20:53:08.174 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server
2023-08-08T20:53:08.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com'
2023-08-08T20:53:08.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-08T20:53:08.259 DEBUG:teuthology.orchestra.run.smithi137:> true
2023-08-08T20:53:08.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com'
2023-08-08T20:53:08.427 DEBUG:teuthology.orchestra.run.smithi137:> uname -m
2023-08-08T20:53:08.482 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64
2023-08-08T20:53:08.482 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release
2023-08-08T20:53:08.537 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream"
2023-08-08T20:53:08.537 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9"
2023-08-08T20:53:08.537 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos"
2023-08-08T20:53:08.537 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora"
2023-08-08T20:53:08.537 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9"
2023-08-08T20:53:08.538 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9"
2023-08-08T20:53:08.538 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9"
2023-08-08T20:53:08.538 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31"
2023-08-08T20:53:08.538 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon"
2023-08-08T20:53:08.538 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-08-08T20:53:08.538 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/"
2023-08-08T20:53:08.539 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-08T20:53:08.539 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-08-08T20:53:08.539 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-08-08T20:53:08.539 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server
2023-08-08T20:53:08.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T20:53:08.644 INFO:teuthology.dispatcher.supervisor:Running job 7363339
2023-08-08T20:53:08.644 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/virtualenv/bin/teuthology -v --owner scheduled_prsrivas@teuthology --archive /home/teuthworker/archive/prsrivas-2023-08-08_16:45:21-rgw-wip-rgw-61916-test-failures-distro-default-smithi/7363339 --name prsrivas-2023-08-08_16:45:21-rgw-wip-rgw-61916-test-failures-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_latest} tasks/test_multi} -- /home/teuthworker/archive/prsrivas-2023-08-08_16:45:21-rgw-wip-rgw-61916-test-failures-distro-default-smithi/7363339/orig.config.yaml
2023-08-08T20:53:08.647 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/prsrivas-2023-08-08_16:45:21-rgw-wip-rgw-61916-test-failures-distro-default-smithi/7363339
2023-08-08T20:53:08.649 INFO:teuthology.dispatcher.supervisor:Job PID: 2679
2023-08-08T20:53:08.649 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-08-08T20:55:08.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T20:57:08.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T20:59:09.001 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:01:09.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:03:09.322 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:05:09.481 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:07:09.637 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:09:09.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:11:09.961 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:13:10.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:15:10.282 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:17:10.445 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:19:10.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:21:10.769 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:23:10.925 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:25:11.089 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:27:11.251 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:29:11.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:31:11.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:33:11.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:35:11.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:37:12.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:39:12.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:41:12.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:43:12.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:45:12.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:47:12.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:49:13.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:51:13.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:53:13.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:55:13.444 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:57:13.613 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T21:59:13.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T22:01:13.957 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T22:03:14.128 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T22:05:14.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T22:07:14.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T22:09:14.645 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T22:11:14.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T22:13:14.969 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-08T22:13:15.031 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-08-08T22:13:15.207 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-08-08T22:13:15.208 INFO:teuthology.nuke:Checking targets against current locks
2023-08-08T22:13:15.256 DEBUG:teuthology.nuke:shortname: smithi090
2023-08-08T22:13:15.257 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-08T22:13:15.280 DEBUG:teuthology.nuke:shortname: smithi137
2023-08-08T22:13:15.281 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-08T22:13:15.285 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/prsrivas-2023-08-08_16:45:21-rgw-wip-rgw-61916-test-failures-distro-default-smithi/7363339', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-08 20:47:28.473907', 'locked_by': 'scheduled_prsrivas@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPkhj1CMOc4iJJ4uy48kbYdMBpXMO0FW+pppgA8mBcmr7TrG6BjG6XiB6bobB50qIT20Bq8w+AFM6cvwlw6h7kI='}
2023-08-08T22:13:15.310 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/prsrivas-2023-08-08_16:45:21-rgw-wip-rgw-61916-test-failures-distro-default-smithi/7363339', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-08 20:47:28.475668', 'locked_by': 'scheduled_prsrivas@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIu1UDtibrWCkcK9AWf3c/ar2a31YdcegVcF9XlqNOP57kDhEG4geBZW9Pquyb5Iv6X9OgJprUmKFgNccPzbguY='}
2023-08-08T22:13:15.335 INFO:teuthology.orchestra.console.smithi090:Power off
2023-08-08T22:13:15.336 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-08T22:13:15.356 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off
2023-08-08T22:13:15.357 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T22:13:15.376 INFO:teuthology.orchestra.console.smithi137:Power off
2023-08-08T22:13:15.376 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-08T22:13:15.387 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off
2023-08-08T22:13:15.401 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off
2023-08-08T22:13:15.401 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-08T22:13:15.424 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off
2023-08-08T22:13:15.489 INFO:teuthology.orchestra.console.smithi090:Power off completed
2023-08-08T22:13:15.527 INFO:teuthology.orchestra.console.smithi137:Power off completed
2023-08-08T22:13:15.639 INFO:teuthology.lock.ops:unlocked: smithi090.front.sepia.ceph.com
2023-08-08T22:13:15.675 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com