2023-08-22T12:47:52.585 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-22T12:47:52.587 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T12:47:52.700 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7271fa9e50>, signals=[15]) 2023-08-22T12:47:52.701 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging 2023-08-22T12:47:52.701 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server 2023-08-22T12:47:52.702 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-08-22T12:47:52.721 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /host {"name": "smithi146"} 2023-08-22T12:47:52.801 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-08-22T12:47:52.851 DEBUG:teuthology.provision.fog.smithi146:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-08-22T12:47:52.852 DEBUG:teuthology.provision.fog.smithi146:FOG request: PUT /host/251 {"imageID": 6} 2023-08-22T12:47:52.904 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of ubuntu 22.04 2023-08-22T12:47:52.905 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:47:53.035 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /tasktype {"name": "deploy"} 2023-08-22T12:47:53.073 DEBUG:teuthology.provision.fog.smithi146:FOG request: POST /host/251/task {"taskTypeID": 1} 2023-08-22T12:47:53.129 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:47:53.289 INFO:teuthology.orchestra.console.smithi146:Power off 2023-08-22T12:47:53.291 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-22T12:47:53.318 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2023-08-22T12:47:53.319 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-22T12:47:53.340 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-08-22T12:47:57.344 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-22T12:47:57.453 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-08-22T12:48:01.456 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-22T12:48:01.567 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-08-22T12:48:01.668 INFO:teuthology.orchestra.console.smithi146:Power off completed 2023-08-22T12:48:01.769 INFO:teuthology.orchestra.console.smithi146:Power on 2023-08-22T12:48:01.769 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-22T12:48:01.811 DEBUG:teuthology.orchestra.console.smithi146:power on output: Chassis Power Control: Up/On 2023-08-22T12:48:01.812 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-22T12:48:01.851 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-08-22T12:48:05.852 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-22T12:48:05.964 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-08-22T12:48:09.968 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-22T12:48:10.079 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-08-22T12:48:10.180 INFO:teuthology.orchestra.console.smithi146:Power on completed 2023-08-22T12:48:10.281 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish 2023-08-22T12:48:10.281 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:48:25.425 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:48:40.560 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:48:55.680 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:49:10.816 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:49:25.936 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:49:41.040 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:49:56.144 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:50:11.248 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:50:26.339 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:50:41.448 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:50:56.524 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:51:11.656 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:51:26.733 DEBUG:teuthology.provision.fog.smithi146:FOG request: GET /task/active None 2023-08-22T12:51:26.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:51:29.876 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-22T12:51:35.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:51:38.964 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-22T12:51:44.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:51:48.052 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-22T12:51:54.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:51:57.136 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-22T12:52:03.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:52:06.192 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-22T12:52:12.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:52:15.252 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-22T12:52:21.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:52:24.340 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-22T12:52:30.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:52:33.428 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-08-22T12:52:39.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:53:39.492 WARNING:teuthology.provision.fog.smithi146:timed out 2023-08-22T12:53:45.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:53:45.699 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-22T12:53:45.701 DEBUG:teuthology.orchestra.run.smithi146:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-22T12:53:46.226 INFO:teuthology.provision.fog.smithi146:Node is ready 2023-08-22T12:53:46.226 DEBUG:teuthology.orchestra.run.smithi146:> hostname 2023-08-22T12:53:46.274 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com 2023-08-22T12:53:46.275 DEBUG:teuthology.orchestra.run.smithi146:> grep smithi146.front.sepia.ceph.com /etc/hosts 2023-08-22T12:53:46.320 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146 2023-08-22T12:53:46.321 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146 2023-08-22T12:53:46.452 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hosts 2023-08-22T12:53:46.466 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/172.21.15.146/172.21.15.146/g' /etc/hosts 2023-08-22T12:53:46.524 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hostname 2023-08-22T12:53:46.581 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146 2023-08-22T12:53:46.634 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-08-22T12:53:46.679 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-08-22T12:53:46.680 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Ubuntu" 2023-08-22T12:53:46.680 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="22.04" 2023-08-22T12:53:46.680 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-08-22T12:53:46.680 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_CODENAME=jammy 2023-08-22T12:53:46.681 INFO:teuthology.orchestra.run.smithi146.stdout:ID=ubuntu 2023-08-22T12:53:46.681 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE=debian 2023-08-22T12:53:46.681 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-22T12:53:46.681 INFO:teuthology.orchestra.run.smithi146.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-22T12:53:46.681 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-22T12:53:46.681 INFO:teuthology.orchestra.run.smithi146.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-22T12:53:46.681 INFO:teuthology.orchestra.run.smithi146.stdout:UBUNTU_CODENAME=jammy 2023-08-22T12:53:46.682 INFO:teuthology.provision.fog.smithi146:Deploy complete! 2023-08-22T12:53:46.684 DEBUG:teuthology.parallel:result is None 2023-08-22T12:53:46.805 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com 2023-08-22T12:53:46.807 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-22T12:53:46.827 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server 2023-08-22T12:53:46.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-08-22T12:53:46.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-22T12:53:46.909 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-08-22T12:53:47.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-08-22T12:53:47.013 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2023-08-22T12:53:47.062 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2023-08-22T12:53:47.063 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-08-22T12:53:47.111 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-08-22T12:53:47.111 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Ubuntu" 2023-08-22T12:53:47.111 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="22.04" 2023-08-22T12:53:47.111 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-08-22T12:53:47.111 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_CODENAME=jammy 2023-08-22T12:53:47.112 INFO:teuthology.orchestra.run.smithi146.stdout:ID=ubuntu 2023-08-22T12:53:47.112 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE=debian 2023-08-22T12:53:47.112 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-22T12:53:47.112 INFO:teuthology.orchestra.run.smithi146.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-22T12:53:47.112 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-22T12:53:47.112 INFO:teuthology.orchestra.run.smithi146.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-22T12:53:47.113 INFO:teuthology.orchestra.run.smithi146.stdout:UBUNTU_CODENAME=jammy 2023-08-22T12:53:47.113 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-08-22T12:53:47.135 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T12:53:47.230 INFO:teuthology.dispatcher.supervisor:Running job 7376239 2023-08-22T12:53:47.230 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/virtualenv/bin/teuthology -v --owner scheduled_dang@sidious.arb.redhat.com --archive /home/teuthworker/archive/dang-2023-08-22_08:07:27-rgw-wip-dang-posix-driver-distro-default-smithi/7376239 --name dang-2023-08-22_08:07:27-rgw-wip-dang-posix-driver-distro-default-smithi --description rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} -- /home/teuthworker/archive/dang-2023-08-22_08:07:27-rgw-wip-dang-posix-driver-distro-default-smithi/7376239/orig.config.yaml 2023-08-22T12:53:47.234 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dang-2023-08-22_08:07:27-rgw-wip-dang-posix-driver-distro-default-smithi/7376239 2023-08-22T12:53:47.236 INFO:teuthology.dispatcher.supervisor:Job PID: 21106 2023-08-22T12:53:47.236 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-22T12:55:47.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T12:57:47.503 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T12:59:47.669 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:01:47.837 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:03:48.001 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:05:48.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:07:48.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:09:48.493 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:11:48.624 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:13:48.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:15:48.949 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:17:49.113 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:19:49.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:21:49.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:23:49.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:25:49.797 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-22T13:25:49.862 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-22T13:25:49.995 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-22T13:25:49.995 INFO:teuthology.nuke:Checking targets against current locks 2023-08-22T13:25:50.044 DEBUG:teuthology.nuke:shortname: smithi146 2023-08-22T13:25:50.044 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-22T13:25:50.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dang-2023-08-22_08:07:27-rgw-wip-dang-posix-driver-distro-default-smithi/7376239', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-22 12:47:51.779817', 'locked_by': 'scheduled_dang@sidious.arb.redhat.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAS1Wl65rcM9TXJQb229Yyny0e1JRz65XuxfQ7fu8pfWj797JOU9ydag3hYkzE+fQElDDsZFJ1gtd40v/a+bdtM='} 2023-08-22T13:25:50.118 INFO:teuthology.orchestra.console.smithi146:Power off 2023-08-22T13:25:50.119 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-22T13:25:50.140 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2023-08-22T13:25:50.141 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-22T13:25:50.162 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-08-22T13:25:50.263 INFO:teuthology.orchestra.console.smithi146:Power off completed 2023-08-22T13:25:50.421 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com