2023-08-08T14:53:16.806 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-08T14:53:16.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T14:53:16.909 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fac743b0280>, signals=[15]) 2023-08-08T14:53:16.912 INFO:teuthology.lock.ops:Start node 'smithi114.front.sepia.ceph.com' reimaging 2023-08-08T14:53:16.913 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: reset os type and version on server 2023-08-08T14:53:16.913 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2023-08-08T14:53:16.933 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /host {"name": "smithi114"} 2023-08-08T14:53:17.007 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-08-08T14:53:17.048 DEBUG:teuthology.provision.fog.smithi114:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-08-08T14:53:17.048 DEBUG:teuthology.provision.fog.smithi114:FOG request: PUT /host/219 {"imageID": 2} 2023-08-08T14:53:17.095 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of ubuntu 20.04 2023-08-08T14:53:17.096 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:53:17.151 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /tasktype {"name": "deploy"} 2023-08-08T14:53:17.187 DEBUG:teuthology.provision.fog.smithi114:FOG request: POST /host/219/task {"taskTypeID": 1} 2023-08-08T14:53:17.255 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:53:17.338 INFO:teuthology.orchestra.console.smithi114:Power off 2023-08-08T14:53:17.339 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T14:53:17.361 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2023-08-08T14:53:17.362 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T14:53:17.387 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2023-08-08T14:53:21.388 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T14:53:21.499 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2023-08-08T14:53:25.504 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T14:53:25.615 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-08-08T14:53:25.716 INFO:teuthology.orchestra.console.smithi114:Power off completed 2023-08-08T14:53:25.817 INFO:teuthology.orchestra.console.smithi114:Power on 2023-08-08T14:53:25.817 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-08T14:53:25.847 DEBUG:teuthology.orchestra.console.smithi114:power on output: Chassis Power Control: Up/On 2023-08-08T14:53:25.848 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T14:53:25.869 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-08-08T14:53:29.872 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T14:53:29.982 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-08-08T14:53:33.984 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T14:53:34.094 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2023-08-08T14:53:34.195 INFO:teuthology.orchestra.console.smithi114:Power on completed 2023-08-08T14:53:34.295 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish 2023-08-08T14:53:34.296 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:53:49.368 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:54:04.440 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:54:19.509 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:54:34.584 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:54:49.668 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:55:04.736 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:55:19.795 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:55:34.864 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:55:49.924 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:56:04.992 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:56:20.072 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:56:35.140 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:56:50.208 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:57:05.272 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:57:20.336 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-08-08T14:57:20.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:57:23.440 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-08-08T14:57:29.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:57:32.496 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-08-08T14:57:38.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:57:41.556 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-08-08T14:57:47.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:57:50.641 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-08-08T14:57:56.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:57:59.728 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-08-08T14:58:05.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:58:08.784 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-08-08T14:58:14.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:59:14.849 WARNING:teuthology.provision.fog.smithi114:timed out 2023-08-08T14:59:20.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:59:21.065 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-08T14:59:21.066 DEBUG:teuthology.orchestra.run.smithi114:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-08T14:59:21.533 INFO:teuthology.provision.fog.smithi114:Node is ready 2023-08-08T14:59:21.534 DEBUG:teuthology.orchestra.run.smithi114:> hostname 2023-08-08T14:59:21.582 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com 2023-08-08T14:59:21.583 DEBUG:teuthology.orchestra.run.smithi114:> grep smithi114.front.sepia.ceph.com /etc/hosts 2023-08-08T14:59:21.590 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114 2023-08-08T14:59:21.591 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114 2023-08-08T14:59:21.679 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi114.front.sepia.ceph.com/smithi114/g' /etc/hosts 2023-08-08T14:59:21.697 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/172.21.15.114/172.21.15.114/g' /etc/hosts 2023-08-08T14:59:21.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi114.front.sepia.ceph.com/smithi114/g' /etc/hostname 2023-08-08T14:59:21.720 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114 2023-08-08T14:59:21.729 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2023-08-08T14:59:21.779 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Ubuntu" 2023-08-08T14:59:21.779 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-08-08T14:59:21.779 INFO:teuthology.orchestra.run.smithi114.stdout:ID=ubuntu 2023-08-08T14:59:21.780 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE=debian 2023-08-08T14:59:21.780 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-08-08T14:59:21.781 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="20.04" 2023-08-08T14:59:21.781 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-08T14:59:21.781 INFO:teuthology.orchestra.run.smithi114.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-08T14:59:21.781 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-08T14:59:21.781 INFO:teuthology.orchestra.run.smithi114.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-08T14:59:21.781 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_CODENAME=focal 2023-08-08T14:59:21.781 INFO:teuthology.orchestra.run.smithi114.stdout:UBUNTU_CODENAME=focal 2023-08-08T14:59:21.782 INFO:teuthology.provision.fog.smithi114:Deploy complete! 2023-08-08T14:59:21.783 DEBUG:teuthology.parallel:result is None 2023-08-08T14:59:21.963 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com 2023-08-08T14:59:21.964 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-08T14:59:22.005 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server 2023-08-08T14:59:22.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-08-08T14:59:22.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-08T14:59:22.058 DEBUG:teuthology.orchestra.run.smithi114:> true 2023-08-08T14:59:22.177 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-08-08T14:59:22.178 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2023-08-08T14:59:22.184 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2023-08-08T14:59:22.184 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2023-08-08T14:59:22.190 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Ubuntu" 2023-08-08T14:59:22.191 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-08-08T14:59:22.191 INFO:teuthology.orchestra.run.smithi114.stdout:ID=ubuntu 2023-08-08T14:59:22.191 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE=debian 2023-08-08T14:59:22.191 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-08-08T14:59:22.191 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="20.04" 2023-08-08T14:59:22.191 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-08T14:59:22.191 INFO:teuthology.orchestra.run.smithi114.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-08T14:59:22.192 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-08T14:59:22.192 INFO:teuthology.orchestra.run.smithi114.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-08T14:59:22.192 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_CODENAME=focal 2023-08-08T14:59:22.192 INFO:teuthology.orchestra.run.smithi114.stdout:UBUNTU_CODENAME=focal 2023-08-08T14:59:22.192 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2023-08-08T14:59:22.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T14:59:22.478 INFO:teuthology.dispatcher.supervisor:Running job 7362812 2023-08-08T14:59:22.479 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-08-08_14:42:45-rgw-wip-yuri3-testing-2023-08-01-0825-pacific-distro-default-smithi/7362812 --name yuriw-2023-08-08_14:42:45-rgw-wip-yuri3-testing-2023-08-01-0825-pacific-distro-default-smithi --description rgw/tempest/{clusters/fixed-1 frontend/civetweb overrides tasks/rgw_tempest ubuntu_latest} -- /home/teuthworker/archive/yuriw-2023-08-08_14:42:45-rgw-wip-yuri3-testing-2023-08-01-0825-pacific-distro-default-smithi/7362812/orig.config.yaml 2023-08-08T14:59:22.482 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-08-08_14:42:45-rgw-wip-yuri3-testing-2023-08-01-0825-pacific-distro-default-smithi/7362812 2023-08-08T14:59:22.483 INFO:teuthology.dispatcher.supervisor:Job PID: 23305 2023-08-08T14:59:22.483 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-08T15:01:22.580 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:03:22.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:05:22.917 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:07:23.077 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:09:23.237 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:11:23.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:13:23.588 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:15:23.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:17:23.913 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:19:24.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:21:24.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:23:24.425 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-08T15:23:24.492 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-08T15:23:24.626 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-08T15:23:24.627 INFO:teuthology.nuke:Checking targets against current locks 2023-08-08T15:23:24.675 DEBUG:teuthology.nuke:shortname: smithi114 2023-08-08T15:23:24.675 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-08T15:23:24.699 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-08_14:42:45-rgw-wip-yuri3-testing-2023-08-01-0825-pacific-distro-default-smithi/7362812', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-08 14:53:16.010991', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-08-08T15:23:24.751 INFO:teuthology.orchestra.console.smithi114:Power off 2023-08-08T15:23:24.752 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-08T15:23:24.773 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2023-08-08T15:23:24.774 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-08T15:23:24.797 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-08-08T15:23:24.899 INFO:teuthology.orchestra.console.smithi114:Power off completed 2023-08-08T15:23:25.049 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com