2023-12-12T16:20:39.032 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-12T16:20:39.033 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:20:39.168 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fb87acb0580>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fb87bbf7940>, signals=[15])
2023-12-12T16:20:39.168 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging
2023-12-12T16:20:39.169 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server
2023-12-12T16:20:39.169 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2023-12-12T16:20:39.300 DEBUG:teuthology.provision.fog.smithi153:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-12-12T16:20:39.344 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of ubuntu 22.04
2023-12-12T16:20:39.710 INFO:teuthology.orchestra.console.smithi153:Power off
2023-12-12T16:20:39.711 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-12T16:20:39.731 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off
2023-12-12T16:20:39.732 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T16:20:39.753 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2023-12-12T16:20:43.754 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T16:20:43.863 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2023-12-12T16:20:47.869 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T16:20:47.977 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2023-12-12T16:20:51.979 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T16:20:52.089 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2023-12-12T16:20:52.189 INFO:teuthology.orchestra.console.smithi153:Power off completed
2023-12-12T16:20:52.290 INFO:teuthology.orchestra.console.smithi153:Power on
2023-12-12T16:20:52.290 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-12T16:20:52.311 DEBUG:teuthology.orchestra.console.smithi153:power on output: Chassis Power Control: Up/On
2023-12-12T16:20:52.311 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T16:20:52.331 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2023-12-12T16:20:56.334 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T16:20:56.443 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2023-12-12T16:20:56.543 INFO:teuthology.orchestra.console.smithi153:Power on completed
2023-12-12T16:20:56.644 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish
2023-12-12T16:27:59.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:28:02.370 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:28:08.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:28:11.426 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:28:17.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:28:20.518 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:28:26.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:28:29.602 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:28:35.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:28:38.658 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:28:44.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:28:47.714 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:28:53.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:28:56.770 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:29:02.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:29:21.122 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:29:27.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:29:30.182 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:29:36.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:29:39.270 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:29:45.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:29:45.276 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:29:51.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:29:51.284 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:29:57.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:29:57.292 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:30:03.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:30:03.300 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-12-12T16:30:09.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:30:09.522 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-12T16:30:09.522 DEBUG:teuthology.orchestra.run.smithi153:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-12T16:30:09.923 INFO:teuthology.provision.fog.smithi153:Node is ready
2023-12-12T16:30:09.924 DEBUG:teuthology.orchestra.run.smithi153:> hostname
2023-12-12T16:30:09.971 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com
2023-12-12T16:30:09.971 DEBUG:teuthology.orchestra.run.smithi153:> grep smithi153.front.sepia.ceph.com /etc/hosts
2023-12-12T16:30:10.020 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153
2023-12-12T16:30:10.020 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname smithi153
2023-12-12T16:30:10.173 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi153.front.sepia.ceph.com/smithi153/g' /etc/hosts
2023-12-12T16:30:10.187 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/172.21.15.153/172.21.15.153/g' /etc/hosts
2023-12-12T16:30:10.243 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi153.front.sepia.ceph.com/smithi153/g' /etc/hostname
2023-12-12T16:30:10.298 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname smithi153
2023-12-12T16:30:10.354 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release
2023-12-12T16:30:10.400 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-12-12T16:30:10.400 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Ubuntu"
2023-12-12T16:30:10.400 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="22.04"
2023-12-12T16:30:10.400 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-12-12T16:30:10.401 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_CODENAME=jammy
2023-12-12T16:30:10.401 INFO:teuthology.orchestra.run.smithi153.stdout:ID=ubuntu
2023-12-12T16:30:10.401 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE=debian
2023-12-12T16:30:10.401 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-12T16:30:10.401 INFO:teuthology.orchestra.run.smithi153.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-12T16:30:10.401 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-12T16:30:10.402 INFO:teuthology.orchestra.run.smithi153.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-12T16:30:10.402 INFO:teuthology.orchestra.run.smithi153.stdout:UBUNTU_CODENAME=jammy
2023-12-12T16:30:10.403 INFO:teuthology.provision.fog.smithi153:Deploy complete!
2023-12-12T16:30:10.404 DEBUG:teuthology.parallel:result is None
2023-12-12T16:30:10.537 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com
2023-12-12T16:30:10.537 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-12T16:30:10.723 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server
2023-12-12T16:30:10.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2023-12-12T16:30:10.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T16:30:10.802 DEBUG:teuthology.orchestra.run.smithi153:> true
2023-12-12T16:30:10.910 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com'
2023-12-12T16:30:10.910 DEBUG:teuthology.orchestra.run.smithi153:> uname -m
2023-12-12T16:30:10.960 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64
2023-12-12T16:30:10.960 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release
2023-12-12T16:30:11.007 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-12-12T16:30:11.008 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Ubuntu"
2023-12-12T16:30:11.008 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="22.04"
2023-12-12T16:30:11.008 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-12-12T16:30:11.009 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_CODENAME=jammy
2023-12-12T16:30:11.009 INFO:teuthology.orchestra.run.smithi153.stdout:ID=ubuntu
2023-12-12T16:30:11.009 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE=debian
2023-12-12T16:30:11.010 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.ubuntu.com/"
2023-12-12T16:30:11.010 INFO:teuthology.orchestra.run.smithi153.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-12-12T16:30:11.010 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-12-12T16:30:11.010 INFO:teuthology.orchestra.run.smithi153.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-12-12T16:30:11.010 INFO:teuthology.orchestra.run.smithi153.stdout:UBUNTU_CODENAME=jammy
2023-12-12T16:30:11.011 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2023-12-12T16:30:11.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:30:11.126 INFO:teuthology.dispatcher.supervisor:Running job 7488654
2023-12-12T16:30:11.126 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-12_16:18:21-rados-wip-yuri-testing-2023-12-11-1524-distro-default-smithi/7488654 --name yuriw-2023-12-12_16:18:21-rados-wip-yuri-testing-2023-12-11-1524-distro-default-smithi --description rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_read} -- /home/teuthworker/archive/yuriw-2023-12-12_16:18:21-rados-wip-yuri-testing-2023-12-11-1524-distro-default-smithi/7488654/orig.config.yaml
2023-12-12T16:30:11.130 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-12-12_16:18:21-rados-wip-yuri-testing-2023-12-11-1524-distro-default-smithi/7488654
2023-12-12T16:30:11.130 INFO:teuthology.dispatcher.supervisor:Job PID: 2708746
2023-12-12T16:30:11.131 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-12T16:32:11.230 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:34:11.398 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:36:11.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:38:11.746 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:40:11.930 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:42:12.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:44:12.258 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:46:12.437 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:48:12.606 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:50:12.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:52:12.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:54:13.143 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:56:13.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T16:58:13.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T17:00:13.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T17:00:13.729 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-12-12T17:00:13.814 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-12T17:00:13.814 INFO:teuthology.nuke:Checking targets against current locks
2023-12-12T17:00:13.863 DEBUG:teuthology.nuke:shortname: smithi153
2023-12-12T17:00:13.863 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-12T17:00:13.885 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-12_16:18:21-rados-wip-yuri-testing-2023-12-11-1524-distro-default-smithi/7488654', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 16:20:38.265452', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRUFt28G4/XD3QmMREAcoaUj1DeyiGzHmnRUYUh3c9EI8Fh0D9453Mz5Vk9I/f9TAx+lyLkg1ZDT1Bz6IN+XJ4='}
2023-12-12T17:00:13.936 INFO:teuthology.orchestra.console.smithi153:Power off
2023-12-12T17:00:13.936 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-12T17:00:13.957 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off
2023-12-12T17:00:13.958 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T17:00:13.981 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2023-12-12T17:00:14.081 INFO:teuthology.orchestra.console.smithi153:Power off completed
2023-12-12T17:00:14.236 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com