2023-08-09T09:40:48.619 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-09T09:40:48.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T09:40:48.728 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd49b2e4f70>, signals=[15]) 2023-08-09T09:40:48.730 INFO:teuthology.lock.ops:Start node 'smithi044.front.sepia.ceph.com' reimaging 2023-08-09T09:40:48.730 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: reset os type and version on server 2023-08-09T09:40:48.730 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2023-08-09T09:40:48.757 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /host {"name": "smithi044"} 2023-08-09T09:40:48.828 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-08-09T09:40:48.868 DEBUG:teuthology.provision.fog.smithi044:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-08-09T09:40:48.868 DEBUG:teuthology.provision.fog.smithi044:FOG request: PUT /host/149 {"imageID": 2} 2023-08-09T09:40:48.915 INFO:teuthology.provision.fog.smithi044:Scheduling deploy of ubuntu 20.04 2023-08-09T09:40:48.916 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:40:48.958 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /tasktype {"name": "deploy"} 2023-08-09T09:40:48.996 DEBUG:teuthology.provision.fog.smithi044:FOG request: POST /host/149/task {"taskTypeID": 1} 2023-08-09T09:40:49.045 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:40:49.134 INFO:teuthology.orchestra.console.smithi044:Power off 2023-08-09T09:40:49.135 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-09T09:40:49.158 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2023-08-09T09:40:49.158 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T09:40:49.179 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-08-09T09:40:53.184 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T09:40:53.293 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-08-09T09:40:57.297 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T09:40:57.408 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-08-09T09:41:01.412 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T09:41:01.525 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-08-09T09:41:01.626 INFO:teuthology.orchestra.console.smithi044:Power off completed 2023-08-09T09:41:01.727 INFO:teuthology.orchestra.console.smithi044:Power on 2023-08-09T09:41:01.727 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-09T09:41:01.753 DEBUG:teuthology.orchestra.console.smithi044:power on output: Chassis Power Control: Up/On 2023-08-09T09:41:01.754 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T09:41:01.777 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-08-09T09:41:05.781 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T09:41:05.890 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-08-09T09:41:05.991 INFO:teuthology.orchestra.console.smithi044:Power on completed 2023-08-09T09:41:06.092 INFO:teuthology.provision.fog.smithi044:Waiting for deploy to finish 2023-08-09T09:41:06.093 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:41:21.168 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:41:36.244 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:41:51.308 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:42:06.377 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:42:21.452 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:42:36.524 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:42:51.596 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:43:06.680 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:43:21.760 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:43:36.836 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:43:51.908 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:44:06.988 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:44:22.083 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:44:37.148 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-09T09:44:37.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:44:40.272 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-09T09:44:46.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:44:49.328 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-09T09:44:55.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:44:58.416 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-09T09:45:04.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:45:07.472 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-09T09:45:13.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:45:16.560 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-09T09:45:22.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:45:25.648 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-09T09:45:31.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:46:31.712 WARNING:teuthology.provision.fog.smithi044:timed out 2023-08-09T09:46:37.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:46:37.921 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-09T09:46:37.922 DEBUG:teuthology.orchestra.run.smithi044:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-09T09:46:38.534 INFO:teuthology.provision.fog.smithi044:Node is ready 2023-08-09T09:46:38.535 DEBUG:teuthology.orchestra.run.smithi044:> hostname 2023-08-09T09:46:38.541 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044.front.sepia.ceph.com 2023-08-09T09:46:38.542 DEBUG:teuthology.orchestra.run.smithi044:> grep smithi044.front.sepia.ceph.com /etc/hosts 2023-08-09T09:46:38.549 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44 smithi044.front.sepia.ceph.com smithi044 2023-08-09T09:46:38.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2023-08-09T09:46:38.679 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hosts 2023-08-09T09:46:38.696 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/172.21.15.44/172.21.15.44/g' /etc/hosts 2023-08-09T09:46:38.711 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hostname 2023-08-09T09:46:38.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2023-08-09T09:46:38.783 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2023-08-09T09:46:38.787 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="Ubuntu" 2023-08-09T09:46:38.787 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-08-09T09:46:38.787 INFO:teuthology.orchestra.run.smithi044.stdout:ID=ubuntu 2023-08-09T09:46:38.787 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE=debian 2023-08-09T09:46:38.787 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-08-09T09:46:38.788 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="20.04" 2023-08-09T09:46:38.788 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-09T09:46:38.788 INFO:teuthology.orchestra.run.smithi044.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-09T09:46:38.788 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-09T09:46:38.788 INFO:teuthology.orchestra.run.smithi044.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-09T09:46:38.788 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_CODENAME=focal 2023-08-09T09:46:38.789 INFO:teuthology.orchestra.run.smithi044.stdout:UBUNTU_CODENAME=focal 2023-08-09T09:46:38.789 INFO:teuthology.provision.fog.smithi044:Deploy complete! 2023-08-09T09:46:38.790 DEBUG:teuthology.parallel:result is None 2023-08-09T09:46:38.977 INFO:teuthology.lock.ops:Checking smithi044.front.sepia.ceph.com 2023-08-09T09:46:38.978 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-09T09:46:39.005 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: set os type and version on server 2023-08-09T09:46:39.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2023-08-09T09:46:39.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T09:46:39.053 DEBUG:teuthology.orchestra.run.smithi044:> true 2023-08-09T09:46:39.172 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2023-08-09T09:46:39.173 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2023-08-09T09:46:39.179 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2023-08-09T09:46:39.179 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2023-08-09T09:46:39.184 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="Ubuntu" 2023-08-09T09:46:39.184 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-08-09T09:46:39.184 INFO:teuthology.orchestra.run.smithi044.stdout:ID=ubuntu 2023-08-09T09:46:39.185 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE=debian 2023-08-09T09:46:39.185 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-08-09T09:46:39.185 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="20.04" 2023-08-09T09:46:39.185 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-09T09:46:39.185 INFO:teuthology.orchestra.run.smithi044.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-09T09:46:39.185 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-09T09:46:39.186 INFO:teuthology.orchestra.run.smithi044.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-09T09:46:39.186 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_CODENAME=focal 2023-08-09T09:46:39.186 INFO:teuthology.orchestra.run.smithi044.stdout:UBUNTU_CODENAME=focal 2023-08-09T09:46:39.186 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2023-08-09T09:46:39.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T09:46:39.327 INFO:teuthology.dispatcher.supervisor:Running job 7363893 2023-08-09T09:46:39.328 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-08-09_01:12:27-rados-wip-yuri5-testing-2023-08-08-0807-quincy-distro-default-smithi/7363893 --name yuriw-2023-08-09_01:12:27-rados-wip-yuri5-testing-2023-08-08-0807-quincy-distro-default-smithi --description rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd-backfill} -- /home/teuthworker/archive/yuriw-2023-08-09_01:12:27-rados-wip-yuri5-testing-2023-08-08-0807-quincy-distro-default-smithi/7363893/orig.config.yaml 2023-08-09T09:46:39.332 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-08-09_01:12:27-rados-wip-yuri5-testing-2023-08-08-0807-quincy-distro-default-smithi/7363893 2023-08-09T09:46:39.333 INFO:teuthology.dispatcher.supervisor:Job PID: 13288 2023-08-09T09:46:39.333 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-09T09:48:39.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T09:50:39.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T09:52:39.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T09:54:39.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T09:56:40.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T09:58:40.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T10:00:40.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T10:02:40.560 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T10:04:40.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T10:06:40.913 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T10:08:41.085 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T10:08:41.340 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-09T10:08:41.457 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-09T10:08:41.458 INFO:teuthology.nuke:Checking targets against current locks 2023-08-09T10:08:41.503 DEBUG:teuthology.nuke:shortname: smithi044 2023-08-09T10:08:41.504 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-09T10:08:41.530 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-09_01:12:27-rados-wip-yuri5-testing-2023-08-08-0807-quincy-distro-default-smithi/7363893', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-09 09:40:47.834446', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-08-09T10:08:41.587 INFO:teuthology.orchestra.console.smithi044:Power off 2023-08-09T10:08:41.588 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-09T10:08:41.614 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2023-08-09T10:08:41.615 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T10:08:41.640 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-08-09T10:08:41.741 INFO:teuthology.orchestra.console.smithi044:Power off completed 2023-08-09T10:08:41.903 INFO:teuthology.lock.ops:unlocked: smithi044.front.sepia.ceph.com