2023-12-04T08:39:31.171 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-04T08:39:31.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T08:39:31.283 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f2fac0f5d90>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f2fad1e8550>, signals=[15])
2023-12-04T08:39:31.283 INFO:teuthology.lock.ops:Start node 'smithi163.front.sepia.ceph.com' reimaging
2023-12-04T08:39:31.283 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: reset os type and version on server
2023-12-04T08:39:31.284 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server
2023-12-04T08:39:31.413 DEBUG:teuthology.provision.fog.smithi163:Requesting image smithi_centos_9.stream (ID 10)
2023-12-04T08:39:31.462 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of centos 9.stream
2023-12-04T08:39:31.871 INFO:teuthology.orchestra.console.smithi163:Power off
2023-12-04T08:39:31.871 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-04T08:39:31.891 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off
2023-12-04T08:39:31.892 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T08:39:31.913 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2023-12-04T08:39:35.914 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T08:39:36.022 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2023-12-04T08:39:40.024 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T08:39:40.133 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2023-12-04T08:39:44.137 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T08:39:44.247 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2023-12-04T08:39:44.348 INFO:teuthology.orchestra.console.smithi163:Power off completed
2023-12-04T08:39:44.448 INFO:teuthology.orchestra.console.smithi163:Power on
2023-12-04T08:39:44.448 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-04T08:39:44.472 DEBUG:teuthology.orchestra.console.smithi163:power on output: Chassis Power Control: Up/On
2023-12-04T08:39:44.472 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T08:39:44.491 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2023-12-04T08:39:48.496 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T08:39:48.607 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2023-12-04T08:39:52.610 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T08:39:52.722 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2023-12-04T08:39:52.823 INFO:teuthology.orchestra.console.smithi163:Power on completed
2023-12-04T08:39:52.923 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish
2023-12-04T08:44:26.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:44:29.858 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:44:35.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:44:38.946 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:44:44.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:44:48.038 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:44:54.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:44:57.122 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:45:03.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:45:06.178 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:45:12.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:45:15.234 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:45:21.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:45:24.322 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:45:30.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:45:33.410 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:45:39.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:45:42.466 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2023-12-04T08:45:48.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:46:48.535 WARNING:teuthology.provision.fog.smithi163:timed out
2023-12-04T08:46:54.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:46:54.880 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-04T08:46:54.880 DEBUG:teuthology.orchestra.run.smithi163:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-04T08:46:55.200 INFO:teuthology.provision.fog.smithi163:Node is ready
2023-12-04T08:46:55.201 DEBUG:teuthology.orchestra.run.smithi163:> hostname
2023-12-04T08:46:55.215 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163.front.sepia.ceph.com
2023-12-04T08:46:55.215 DEBUG:teuthology.orchestra.run.smithi163:> grep smithi163.front.sepia.ceph.com /etc/hosts
2023-12-04T08:46:55.270 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163
2023-12-04T08:46:55.271 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname smithi163
2023-12-04T08:46:55.382 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163.front.sepia.ceph.com/smithi163/g' /etc/hosts
2023-12-04T08:46:55.410 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/172.21.15.163/172.21.15.163/g' /etc/hosts
2023-12-04T08:46:55.478 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163.front.sepia.ceph.com/smithi163/g' /etc/hostname
2023-12-04T08:46:55.545 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname smithi163
2023-12-04T08:46:55.611 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release
2023-12-04T08:46:55.665 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-04T08:46:55.666 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-12-04T08:46:55.667 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-04T08:46:55.667 INFO:teuthology.provision.fog.smithi163:Deploy complete!
2023-12-04T08:46:55.668 DEBUG:teuthology.parallel:result is None
2023-12-04T08:46:55.784 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com
2023-12-04T08:46:55.785 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-04T08:46:55.814 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server
2023-12-04T08:46:55.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com'
2023-12-04T08:46:55.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-04T08:46:55.897 DEBUG:teuthology.orchestra.run.smithi163:> true
2023-12-04T08:46:55.968 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com'
2023-12-04T08:46:55.968 DEBUG:teuthology.orchestra.run.smithi163:> uname -m
2023-12-04T08:46:56.022 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64
2023-12-04T08:46:56.022 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release
2023-12-04T08:46:56.076 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-12-04T08:46:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-04T08:46:56.078 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server
2023-12-04T08:46:56.105 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T08:46:58.032 INFO:teuthology.dispatcher.supervisor:Running job 7476777
2023-12-04T08:46:58.033 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-12-04_08:39:13-crimson-rados-wip-matanb-crimson-submit-error-cleanup-testing-distro-crimson-smithi/7476777 --name matan-2023-12-04_08:39:13-crimson-rados-wip-matanb-crimson-submit-error-cleanup-testing-distro-crimson-smithi --description crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_cls_tests} -- /home/teuthworker/archive/matan-2023-12-04_08:39:13-crimson-rados-wip-matanb-crimson-submit-error-cleanup-testing-distro-crimson-smithi/7476777/orig.config.yaml
2023-12-04T08:46:58.036 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/matan-2023-12-04_08:39:13-crimson-rados-wip-matanb-crimson-submit-error-cleanup-testing-distro-crimson-smithi/7476777
2023-12-04T08:46:58.036 INFO:teuthology.dispatcher.supervisor:Job PID: 978144
2023-12-04T08:46:58.036 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-04T08:48:58.135 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T08:50:58.318 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T08:52:58.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T08:54:58.696 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T08:56:58.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T08:58:59.071 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T09:00:59.245 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T09:02:59.414 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T09:04:59.574 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T09:06:59.734 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T09:08:59.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T09:11:00.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-04T09:11:00.134 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-12-04T09:11:00.191 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-04T09:11:00.191 INFO:teuthology.nuke:Checking targets against current locks
2023-12-04T09:11:00.235 DEBUG:teuthology.nuke:shortname: smithi163
2023-12-04T09:11:00.236 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-04T09:11:00.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-12-04_08:39:13-crimson-rados-wip-matanb-crimson-submit-error-cleanup-testing-distro-crimson-smithi/7476777', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-04 08:39:30.428642', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPX7qUSocR6K7qTldK6Iwx66G+rnysKaLfsnbX6NzFMa4cPwcP1l6W4ccMLyllB3UaMr/XO0pfIum7mZfBinUOc='}
2023-12-04T09:11:00.304 INFO:teuthology.orchestra.console.smithi163:Power off
2023-12-04T09:11:00.304 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-04T09:11:00.324 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off
2023-12-04T09:11:00.324 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-04T09:11:00.344 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2023-12-04T09:11:00.444 INFO:teuthology.orchestra.console.smithi163:Power off completed
2023-12-04T09:11:00.592 INFO:teuthology.lock.ops:unlocked: smithi163.front.sepia.ceph.com