2023-10-13T10:52:08.889 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07
2023-10-13T10:52:08.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T10:52:09.023 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f3c048c8b80>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f3c053ba670>, signals=[15])
2023-10-13T10:52:09.024 INFO:teuthology.lock.ops:Start node 'smithi007.front.sepia.ceph.com' reimaging
2023-10-13T10:52:09.024 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: reset os type and version on server
2023-10-13T10:52:09.025 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server
2023-10-13T10:52:09.047 INFO:teuthology.lock.ops:Start node 'smithi151.front.sepia.ceph.com' reimaging
2023-10-13T10:52:09.047 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: reset os type and version on server
2023-10-13T10:52:09.048 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server
2023-10-13T10:52:09.049 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /host {"name": "smithi007"}
2023-10-13T10:52:09.071 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /host {"name": "smithi151"}
2023-10-13T10:52:09.109 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-10-13T10:52:09.133 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-10-13T10:52:09.154 DEBUG:teuthology.provision.fog.smithi007:Requesting image smithi_centos_9.stream (ID 10)
2023-10-13T10:52:09.155 DEBUG:teuthology.provision.fog.smithi007:FOG request: PUT /host/113 {"imageID": 10}
2023-10-13T10:52:09.171 DEBUG:teuthology.provision.fog.smithi151:Requesting image smithi_centos_9.stream (ID 10)
2023-10-13T10:52:09.172 DEBUG:teuthology.provision.fog.smithi151:FOG request: PUT /host/256 {"imageID": 10}
2023-10-13T10:52:09.204 INFO:teuthology.provision.fog.smithi007:Scheduling deploy of centos 9.stream
2023-10-13T10:52:09.205 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:52:09.213 INFO:teuthology.provision.fog.smithi151:Scheduling deploy of centos 9.stream
2023-10-13T10:52:09.213 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:52:09.251 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /tasktype {"name": "deploy"}
2023-10-13T10:52:09.254 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /tasktype {"name": "deploy"}
2023-10-13T10:52:09.286 DEBUG:teuthology.provision.fog.smithi151:FOG request: POST /host/256/task {"taskTypeID": 1}
2023-10-13T10:52:09.293 DEBUG:teuthology.provision.fog.smithi007:FOG request: POST /host/113/task {"taskTypeID": 1}
2023-10-13T10:52:09.350 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:52:09.356 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:52:09.427 INFO:teuthology.orchestra.console.smithi151:Power off
2023-10-13T10:52:09.428 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-13T10:52:09.441 INFO:teuthology.orchestra.console.smithi007:Power off
2023-10-13T10:52:09.442 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-13T10:52:09.450 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off
2023-10-13T10:52:09.451 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T10:52:09.462 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off
2023-10-13T10:52:09.463 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T10:52:09.471 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off
2023-10-13T10:52:09.483 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off
2023-10-13T10:52:09.572 INFO:teuthology.orchestra.console.smithi151:Power off completed
2023-10-13T10:52:09.584 INFO:teuthology.orchestra.console.smithi007:Power off completed
2023-10-13T10:52:09.674 INFO:teuthology.orchestra.console.smithi151:Power on
2023-10-13T10:52:09.674 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-10-13T10:52:09.685 INFO:teuthology.orchestra.console.smithi007:Power on
2023-10-13T10:52:09.685 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-10-13T10:52:09.695 DEBUG:teuthology.orchestra.console.smithi151:power on output: Chassis Power Control: Up/On
2023-10-13T10:52:09.696 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T10:52:09.706 DEBUG:teuthology.orchestra.console.smithi007:power on output: Chassis Power Control: Up/On
2023-10-13T10:52:09.707 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T10:52:09.715 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off
2023-10-13T10:52:09.728 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off
2023-10-13T10:52:13.720 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T10:52:13.729 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T10:52:13.828 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off
2023-10-13T10:52:13.836 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off
2023-10-13T10:52:17.831 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T10:52:17.838 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T10:52:17.941 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on
2023-10-13T10:52:17.947 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on
2023-10-13T10:52:18.042 INFO:teuthology.orchestra.console.smithi151:Power on completed
2023-10-13T10:52:18.049 INFO:teuthology.orchestra.console.smithi007:Power on completed
2023-10-13T10:52:18.143 INFO:teuthology.provision.fog.smithi151:Waiting for deploy to finish
2023-10-13T10:52:18.144 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:52:18.150 INFO:teuthology.provision.fog.smithi007:Waiting for deploy to finish
2023-10-13T10:52:18.150 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:52:33.226 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:52:33.227 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:52:48.309 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:52:48.311 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:53:03.390 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:53:03.391 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:53:18.472 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:53:18.473 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:53:33.554 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:53:33.597 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:53:48.639 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:53:48.660 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:54:03.718 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:54:03.722 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:54:18.821 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:54:18.823 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:54:33.909 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:54:33.911 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:54:48.983 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:54:48.985 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:55:04.060 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:55:04.062 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-10-13T10:55:04.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:55:19.133 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None
2023-10-13T10:55:19.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:55:22.491 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-10-13T10:55:28.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:55:31.578 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-10-13T10:55:37.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:55:37.594 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7
2023-10-13T10:55:40.634 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-10-13T10:55:43.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:55:46.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:55:46.650 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7
2023-10-13T10:55:49.690 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-10-13T10:55:52.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:55:55.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:55:55.710 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7
2023-10-13T10:55:58.746 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-10-13T10:56:01.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:56:04.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:56:07.834 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-10-13T10:56:13.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:57:01.743 WARNING:teuthology.provision.fog.smithi007:timed out
2023-10-13T10:57:07.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:57:07.988 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-13T10:57:07.989 DEBUG:teuthology.orchestra.run.smithi007:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-13T10:57:08.325 INFO:teuthology.provision.fog.smithi007:Node is ready
2023-10-13T10:57:08.325 DEBUG:teuthology.orchestra.run.smithi007:> hostname
2023-10-13T10:57:08.340 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007.front.sepia.ceph.com
2023-10-13T10:57:08.340 DEBUG:teuthology.orchestra.run.smithi007:> grep smithi007.front.sepia.ceph.com /etc/hosts
2023-10-13T10:57:08.396 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7 smithi007.front.sepia.ceph.com smithi007
2023-10-13T10:57:08.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname smithi007
2023-10-13T10:57:08.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/smithi007.front.sepia.ceph.com/smithi007/g' /etc/hosts
2023-10-13T10:57:08.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/172.21.15.7/172.21.15.7/g' /etc/hosts
2023-10-13T10:57:08.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/smithi007.front.sepia.ceph.com/smithi007/g' /etc/hostname
2023-10-13T10:57:08.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname smithi007
2023-10-13T10:57:08.746 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release
2023-10-13T10:57:08.801 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream"
2023-10-13T10:57:08.801 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9"
2023-10-13T10:57:08.802 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos"
2023-10-13T10:57:08.802 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora"
2023-10-13T10:57:08.802 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9"
2023-10-13T10:57:08.802 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9"
2023-10-13T10:57:08.802 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9"
2023-10-13T10:57:08.803 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31"
2023-10-13T10:57:08.803 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon"
2023-10-13T10:57:08.803 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-10-13T10:57:08.803 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/"
2023-10-13T10:57:08.803 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-13T10:57:08.803 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-10-13T10:57:08.804 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-13T10:57:08.804 INFO:teuthology.provision.fog.smithi007:Deploy complete!
2023-10-13T10:57:08.806 DEBUG:teuthology.parallel:result is None
2023-10-13T10:57:13.850 WARNING:teuthology.provision.fog.smithi151:timed out
2023-10-13T10:57:19.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:57:20.089 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-13T10:57:20.089 DEBUG:teuthology.orchestra.run.smithi151:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-13T10:57:20.454 INFO:teuthology.provision.fog.smithi151:Node is ready
2023-10-13T10:57:20.454 DEBUG:teuthology.orchestra.run.smithi151:> hostname
2023-10-13T10:57:20.469 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151.front.sepia.ceph.com
2023-10-13T10:57:20.469 DEBUG:teuthology.orchestra.run.smithi151:> grep smithi151.front.sepia.ceph.com /etc/hosts
2023-10-13T10:57:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151 smithi151.front.sepia.ceph.com smithi151
2023-10-13T10:57:20.524 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151
2023-10-13T10:57:20.628 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi151.front.sepia.ceph.com/smithi151/g' /etc/hosts
2023-10-13T10:57:20.656 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/172.21.15.151/172.21.15.151/g' /etc/hosts
2023-10-13T10:57:20.722 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi151.front.sepia.ceph.com/smithi151/g' /etc/hostname
2023-10-13T10:57:20.789 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151
2023-10-13T10:57:20.856 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release
2023-10-13T10:57:20.909 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream"
2023-10-13T10:57:20.910 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9"
2023-10-13T10:57:20.910 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos"
2023-10-13T10:57:20.910 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora"
2023-10-13T10:57:20.910 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9"
2023-10-13T10:57:20.911 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9"
2023-10-13T10:57:20.911 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9"
2023-10-13T10:57:20.911 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31"
2023-10-13T10:57:20.911 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon"
2023-10-13T10:57:20.911 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-10-13T10:57:20.912 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/"
2023-10-13T10:57:20.912 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-13T10:57:20.912 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-10-13T10:57:20.912 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-13T10:57:20.912 INFO:teuthology.provision.fog.smithi151:Deploy complete!
2023-10-13T10:57:20.913 DEBUG:teuthology.parallel:result is None
2023-10-13T10:57:21.078 INFO:teuthology.lock.ops:Checking smithi007.front.sepia.ceph.com
2023-10-13T10:57:21.079 INFO:teuthology.lock.ops:New key found. Updating...
2023-10-13T10:57:21.105 INFO:teuthology.lock.ops:Checking smithi151.front.sepia.ceph.com
2023-10-13T10:57:21.105 INFO:teuthology.lock.ops:New key found. Updating...
2023-10-13T10:57:21.123 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: set os type and version on server
2023-10-13T10:57:21.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com'
2023-10-13T10:57:21.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:57:21.220 DEBUG:teuthology.orchestra.run.smithi007:> true
2023-10-13T10:57:21.396 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com'
2023-10-13T10:57:21.397 DEBUG:teuthology.orchestra.run.smithi007:> uname -m
2023-10-13T10:57:21.451 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64
2023-10-13T10:57:21.452 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release
2023-10-13T10:57:21.507 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream"
2023-10-13T10:57:21.507 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9"
2023-10-13T10:57:21.507 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos"
2023-10-13T10:57:21.507 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora"
2023-10-13T10:57:21.508 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9"
2023-10-13T10:57:21.508 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9"
2023-10-13T10:57:21.508 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9"
2023-10-13T10:57:21.508 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31"
2023-10-13T10:57:21.509 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon"
2023-10-13T10:57:21.509 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-10-13T10:57:21.509 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/"
2023-10-13T10:57:21.509 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-13T10:57:21.510 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-10-13T10:57:21.510 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-13T10:57:21.510 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server
2023-10-13T10:57:21.538 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: set os type and version on server
2023-10-13T10:57:21.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com'
2023-10-13T10:57:21.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-13T10:57:21.621 DEBUG:teuthology.orchestra.run.smithi151:> true
2023-10-13T10:57:21.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com'
2023-10-13T10:57:21.646 DEBUG:teuthology.orchestra.run.smithi151:> uname -m
2023-10-13T10:57:21.700 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64
2023-10-13T10:57:21.700 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release
2023-10-13T10:57:21.755 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream"
2023-10-13T10:57:21.755 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9"
2023-10-13T10:57:21.755 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos"
2023-10-13T10:57:21.756 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora"
2023-10-13T10:57:21.756 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9"
2023-10-13T10:57:21.756 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9"
2023-10-13T10:57:21.756 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9"
2023-10-13T10:57:21.756 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31"
2023-10-13T10:57:21.757 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon"
2023-10-13T10:57:21.757 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-10-13T10:57:21.757 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/"
2023-10-13T10:57:21.757 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-13T10:57:21.757 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-10-13T10:57:21.758 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-10-13T10:57:21.758 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server
2023-10-13T10:57:21.780 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T10:57:21.871 INFO:teuthology.dispatcher.supervisor:Running job 7424162
2023-10-13T10:57:21.871 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner mchangir --archive /home/teuthworker/archive/mchangir-2023-10-13_10:50:17-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7424162 --name mchangir-2023-10-13_10:50:17-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} -- /home/teuthworker/archive/mchangir-2023-10-13_10:50:17-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7424162/orig.config.yaml
2023-10-13T10:57:21.875 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2023-10-13_10:50:17-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7424162
2023-10-13T10:57:21.876 INFO:teuthology.dispatcher.supervisor:Job PID: 457199
2023-10-13T10:57:21.876 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-10-13T10:59:21.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:01:22.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:03:22.312 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:05:22.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:07:22.660 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:09:22.782 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:11:22.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:13:23.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:15:23.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:17:23.669 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:19:23.788 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:21:23.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:23:24.024 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:25:24.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:27:24.483 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:29:24.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:31:24.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:33:24.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:35:24.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:37:25.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:39:25.178 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:41:25.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:43:25.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:45:25.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:47:25.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:49:25.973 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:51:26.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:53:26.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:55:26.387 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:57:26.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T11:59:26.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:01:26.822 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:03:26.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:05:26.990 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:07:27.118 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:09:27.270 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:11:27.430 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:13:27.644 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:15:27.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:17:27.962 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:19:28.075 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:21:28.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:23:28.368 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:25:28.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:27:28.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:29:28.857 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:31:29.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:33:29.107 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:35:29.190 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:37:29.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:39:29.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:41:29.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:43:29.952 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:45:30.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:47:30.292 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:49:30.464 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:51:30.629 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:53:30.800 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:55:30.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:57:31.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-13T12:57:31.118 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-10-13T12:57:31.277 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-10-13T12:57:31.277 INFO:teuthology.nuke:Checking targets against current locks
2023-10-13T12:57:31.326 DEBUG:teuthology.nuke:shortname: smithi007
2023-10-13T12:57:31.327 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-10-13T12:57:31.343 DEBUG:teuthology.nuke:shortname: smithi151
2023-10-13T12:57:31.343 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-10-13T12:57:31.349 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-13_10:50:17-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7424162', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-13 10:52:08.163791', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLPIngo1R/g1F/Hs17GuVG+sOeivyW/CNbHXVaxPmXyalxBZE70oqSJbZjirVMggEBmWV5KBe4tzOakSAZoaUS4='}
2023-10-13T12:57:31.367 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-13_10:50:17-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7424162', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-13 10:52:08.162148', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIefhSaB581Hlfi8AW/sxd0ybtidteMvDFDz0akui75U1ETbH9lBO9rxiU1+HT3YSycMY0WD4n89Ir4Y9bk1IJM='}
2023-10-13T12:57:31.401 INFO:teuthology.orchestra.console.smithi007:Power off
2023-10-13T12:57:31.401 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-13T12:57:31.415 INFO:teuthology.orchestra.console.smithi151:Power off
2023-10-13T12:57:31.416 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-13T12:57:31.424 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off
2023-10-13T12:57:31.424 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T12:57:31.436 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off
2023-10-13T12:57:31.437 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-13T12:57:31.446 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off
2023-10-13T12:57:31.457 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off
2023-10-13T12:57:31.547 INFO:teuthology.orchestra.console.smithi007:Power off completed
2023-10-13T12:57:31.558 INFO:teuthology.orchestra.console.smithi151:Power off completed
2023-10-13T12:57:31.700 INFO:teuthology.lock.ops:unlocked: smithi007.front.sepia.ceph.com
2023-10-13T12:57:31.708 INFO:teuthology.lock.ops:unlocked: smithi151.front.sepia.ceph.com