2023-08-24T09:52:55.973 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc
2023-08-24T09:52:55.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T09:52:56.088 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f88b3ca3820>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f88b35c2e50>, signals=[15])
2023-08-24T09:52:56.090 INFO:teuthology.lock.ops:Start node 'smithi131.front.sepia.ceph.com' reimaging
2023-08-24T09:52:56.091 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: reset os type and version on server
2023-08-24T09:52:56.091 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server
2023-08-24T09:52:56.121 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /host {"name": "smithi131"}
2023-08-24T09:52:56.201 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-08-24T09:52:56.246 DEBUG:teuthology.provision.fog.smithi131:Requesting image smithi_centos_8.stream (ID 3)
2023-08-24T09:52:56.246 DEBUG:teuthology.provision.fog.smithi131:FOG request: PUT /host/236 {"imageID": 3}
2023-08-24T09:52:56.286 INFO:teuthology.provision.fog.smithi131:Scheduling deploy of centos 8.stream
2023-08-24T09:52:56.286 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:52:56.339 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /tasktype {"name": "deploy"}
2023-08-24T09:52:56.372 DEBUG:teuthology.provision.fog.smithi131:FOG request: POST /host/236/task {"taskTypeID": 1}
2023-08-24T09:52:56.432 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:52:56.530 INFO:teuthology.orchestra.console.smithi131:Power off
2023-08-24T09:52:56.532 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-24T09:52:56.553 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off
2023-08-24T09:52:56.554 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-24T09:52:56.579 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on
2023-08-24T09:53:00.584 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-24T09:53:00.693 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on
2023-08-24T09:53:04.696 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-24T09:53:04.807 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on
2023-08-24T09:53:08.811 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-24T09:53:08.924 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off
2023-08-24T09:53:09.026 INFO:teuthology.orchestra.console.smithi131:Power off completed
2023-08-24T09:53:09.127 INFO:teuthology.orchestra.console.smithi131:Power on
2023-08-24T09:53:09.127 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-24T09:53:09.152 DEBUG:teuthology.orchestra.console.smithi131:power on output: Chassis Power Control: Up/On
2023-08-24T09:53:09.153 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-24T09:53:09.177 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off
2023-08-24T09:53:13.181 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-24T09:53:13.293 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on
2023-08-24T09:53:13.394 INFO:teuthology.orchestra.console.smithi131:Power on completed
2023-08-24T09:53:13.495 INFO:teuthology.provision.fog.smithi131:Waiting for deploy to finish
2023-08-24T09:53:13.495 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:53:28.572 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:53:43.664 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:53:58.744 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:54:13.827 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:54:28.904 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:54:44.006 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:54:59.092 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:55:14.209 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:55:29.320 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:55:44.416 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None
2023-08-24T09:55:44.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-24T09:55:44.569 ERROR:teuthology.orchestra.connection:Error authenticating with smithi131.front.sepia.ceph.com: Authentication failed.
2023-08-24T09:55:44.571 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-24T09:55:44.571 DEBUG:teuthology.orchestra.run.smithi131:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-24T09:55:44.572 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-24T09:57:35.472 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-08-24T09:57:35.485 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi131
2023-08-24T09:57:44.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com'
2023-08-24T09:57:44.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-24T09:57:44.772 DEBUG:teuthology.orchestra.run.smithi131:> true
2023-08-24T09:57:45.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com'
2023-08-24T09:57:45.013 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-24T09:57:45.014 DEBUG:teuthology.orchestra.run.smithi131:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-24T09:57:45.031 INFO:teuthology.provision.fog.smithi131:Node is ready
2023-08-24T09:57:45.032 DEBUG:teuthology.orchestra.run.smithi131:> hostname
2023-08-24T09:57:45.090 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131.front.sepia.ceph.com
2023-08-24T09:57:45.091 DEBUG:teuthology.orchestra.run.smithi131:> grep smithi131.front.sepia.ceph.com /etc/hosts
2023-08-24T09:57:45.149 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131 smithi131.front.sepia.ceph.com smithi131
2023-08-24T09:57:45.150 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131
2023-08-24T09:57:45.250 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hosts
2023-08-24T09:57:45.278 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/172.21.15.131/172.21.15.131/g' /etc/hosts
2023-08-24T09:57:45.346 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hostname
2023-08-24T09:57:45.417 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131
2023-08-24T09:57:45.485 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release
2023-08-24T09:57:45.543 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream"
2023-08-24T09:57:45.544 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8"
2023-08-24T09:57:45.544 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos"
2023-08-24T09:57:45.544 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora"
2023-08-24T09:57:45.544 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8"
2023-08-24T09:57:45.544 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8"
2023-08-24T09:57:45.546 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 8"
2023-08-24T09:57:45.546 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31"
2023-08-24T09:57:45.546 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-08-24T09:57:45.547 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/"
2023-08-24T09:57:45.547 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-24T09:57:45.547 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-08-24T09:57:45.547 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-08-24T09:57:45.549 INFO:teuthology.provision.fog.smithi131:Deploy complete!
2023-08-24T09:57:45.550 DEBUG:teuthology.parallel:result is None
2023-08-24T09:57:45.661 INFO:teuthology.lock.ops:Checking smithi131.front.sepia.ceph.com
2023-08-24T09:57:45.662 INFO:teuthology.lock.ops:New key found. Updating...
2023-08-24T09:57:45.684 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: set os type and version on server
2023-08-24T09:57:45.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com'
2023-08-24T09:57:45.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-24T09:57:45.751 DEBUG:teuthology.orchestra.run.smithi131:> true
2023-08-24T09:57:45.818 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com'
2023-08-24T09:57:45.818 DEBUG:teuthology.orchestra.run.smithi131:> uname -m
2023-08-24T09:57:45.876 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64
2023-08-24T09:57:45.877 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release
2023-08-24T09:57:45.935 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream"
2023-08-24T09:57:45.936 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8"
2023-08-24T09:57:45.936 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos"
2023-08-24T09:57:45.936 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora"
2023-08-24T09:57:45.936 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8"
2023-08-24T09:57:45.936 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8"
2023-08-24T09:57:45.936 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 8"
2023-08-24T09:57:45.937 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31"
2023-08-24T09:57:45.937 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-08-24T09:57:45.937 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/"
2023-08-24T09:57:45.937 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-24T09:57:45.937 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-08-24T09:57:45.937 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-08-24T09:57:45.938 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server
2023-08-24T09:57:45.957 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T09:57:46.059 INFO:teuthology.dispatcher.supervisor:Running job 7378508
2023-08-24T09:57:46.059 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/virtualenv/bin/teuthology -v --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378508 --name matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi --description crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_rw} -- /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378508/orig.config.yaml
2023-08-24T09:57:46.064 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378508
2023-08-24T09:57:46.064 INFO:teuthology.dispatcher.supervisor:Job PID: 3612
2023-08-24T09:57:46.065 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-08-24T09:59:46.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:01:46.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:03:46.417 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:05:46.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:07:46.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:09:46.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:11:47.093 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:13:47.265 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:15:47.437 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:17:47.601 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:19:47.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:21:47.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:23:48.105 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:25:48.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:27:48.441 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-24T10:27:48.515 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-08-24T10:27:48.661 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-08-24T10:27:48.661 INFO:teuthology.nuke:Checking targets against current locks
2023-08-24T10:27:48.702 DEBUG:teuthology.nuke:shortname: smithi131
2023-08-24T10:27:48.703 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-24T10:27:48.728 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378508', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-24 09:52:55.122391', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXbpAFuZQBDzzweEc7Iu2VLaOkyW4gU2z9DiPDzy0KtrruuPyJQLseUkrIyVWKIwU+ImO9h2xbQqQxRQ2Q3PVxkbX0dvL0FtqdLlqKY9yvhwOWLLDlG1GpnMY/aIwgw70c8Kp+CgzDcA5KVej+50JjkFWQ9Roedtx0Vhu14/hUDjkczTvb3tT0vP+b16/Mo2bk0CSwuIZI+elFVHwTkM4baeeLguGig+wnEFRnAb0E4MEwD3xGjxkMKLhjpYYu2ddIdjhOVXkqcyEk0+LHTApsTK2XMJ6KOpxjbI4AEXWpRaZ5uS1dYABVV43Bt2Ni+afT5oE/OY1y7hlEXY7hcMWZMH+SK9RWZFk6mj0EuCSPAFoX5uKOSbIbHrC2LkLNtqC3nvRT/iWq5PHDRsCj8YRBXqG4l8ZsiFbmr7WcI0qzfU1bhff/WDo3rk5+Zt26ApS+3dE5tX2uQTj8tw1ZULGu65hwgH4z+MJYrXiZrOGbgd7j2jOvQVkr9OoekrzQKI0='}
2023-08-24T10:27:48.781 INFO:teuthology.orchestra.console.smithi131:Power off
2023-08-24T10:27:48.782 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-24T10:27:48.808 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off
2023-08-24T10:27:48.808 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-24T10:27:48.832 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off
2023-08-24T10:27:48.933 INFO:teuthology.orchestra.console.smithi131:Power off completed
2023-08-24T10:27:49.080 INFO:teuthology.lock.ops:unlocked: smithi131.front.sepia.ceph.com