2024-01-27T00:51:35.087 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f
2024-01-27T00:51:35.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T00:51:35.194 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fa5b86cef40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fa5b9634c10>, signals=[15])
2024-01-27T00:51:35.194 INFO:teuthology.lock.ops:Start node 'smithi055.front.sepia.ceph.com' reimaging
2024-01-27T00:51:35.194 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: reset os type and version on server
2024-01-27T00:51:35.194 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server
2024-01-27T00:51:35.338 DEBUG:teuthology.provision.fog.smithi055:Requesting image smithi_rhel_8.6 (ID 5)
2024-01-27T00:51:35.383 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of rhel 8.6
2024-01-27T00:51:35.616 INFO:teuthology.orchestra.console.smithi055:Power off
2024-01-27T00:51:35.616 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-27T00:51:35.639 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off
2024-01-27T00:51:35.639 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:51:35.660 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on
2024-01-27T00:51:39.664 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:51:39.774 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on
2024-01-27T00:51:43.777 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:51:43.886 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off
2024-01-27T00:51:43.987 INFO:teuthology.orchestra.console.smithi055:Power off completed
2024-01-27T00:51:44.087 INFO:teuthology.orchestra.console.smithi055:Power on
2024-01-27T00:51:44.087 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-27T00:51:44.109 DEBUG:teuthology.orchestra.console.smithi055:power on output: Chassis Power Control: Up/On
2024-01-27T00:51:44.109 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:51:44.129 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off
2024-01-27T00:51:48.132 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:51:48.240 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on
2024-01-27T00:51:48.340 INFO:teuthology.orchestra.console.smithi055:Power on completed
2024-01-27T00:51:48.441 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish
2024-01-27T00:54:19.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:54:37.888 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55
2024-01-27T00:54:43.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:54:46.972 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55
2024-01-27T00:54:52.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:54:56.028 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55
2024-01-27T00:55:02.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:55:05.084 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55
2024-01-27T00:55:11.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:56:11.147 WARNING:teuthology.provision.fog.smithi055:timed out
2024-01-27T00:56:17.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:56:17.452 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-27T00:56:17.452 DEBUG:teuthology.orchestra.run.smithi055:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-27T00:56:17.723 INFO:teuthology.provision.fog.smithi055:Node is ready
2024-01-27T00:56:17.723 DEBUG:teuthology.orchestra.run.smithi055:> hostname
2024-01-27T00:56:17.744 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055.front.sepia.ceph.com
2024-01-27T00:56:17.744 DEBUG:teuthology.orchestra.run.smithi055:> grep smithi055.front.sepia.ceph.com /etc/hosts
2024-01-27T00:56:17.807 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55 smithi055.front.sepia.ceph.com smithi055
2024-01-27T00:56:17.807 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname smithi055
2024-01-27T00:56:17.940 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi055.front.sepia.ceph.com/smithi055/g' /etc/hosts
2024-01-27T00:56:17.976 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/172.21.15.55/172.21.15.55/g' /etc/hosts
2024-01-27T00:56:18.051 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi055.front.sepia.ceph.com/smithi055/g' /etc/hostname
2024-01-27T00:56:18.126 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname smithi055
2024-01-27T00:56:18.200 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Red Hat Enterprise Linux"
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8.6 (Ootpa)"
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:ID="rhel"
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="fedora"
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8.6"
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8"
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31"
2024-01-27T00:56:18.262 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-01-27T00:56:18.263 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.redhat.com/"
2024-01-27T00:56:18.263 INFO:teuthology.orchestra.run.smithi055.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-01-27T00:56:18.263 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-27T00:56:18.263 INFO:teuthology.orchestra.run.smithi055.stdout:
2024-01-27T00:56:18.263 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-27T00:56:18.263 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-01-27T00:56:18.263 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-01-27T00:56:18.263 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-01-27T00:56:18.264 INFO:teuthology.provision.fog.smithi055:Deploy complete!
2024-01-27T00:56:18.265 DEBUG:teuthology.parallel:result is None
2024-01-27T00:56:18.384 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com
2024-01-27T00:56:18.384 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-27T00:56:18.406 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server
2024-01-27T00:56:18.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com'
2024-01-27T00:56:18.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:56:18.481 DEBUG:teuthology.orchestra.run.smithi055:> true
2024-01-27T00:56:18.553 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com'
2024-01-27T00:56:18.554 DEBUG:teuthology.orchestra.run.smithi055:> uname -m
2024-01-27T00:56:18.614 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64
2024-01-27T00:56:18.614 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release
2024-01-27T00:56:18.675 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Red Hat Enterprise Linux"
2024-01-27T00:56:18.675 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8.6 (Ootpa)"
2024-01-27T00:56:18.675 INFO:teuthology.orchestra.run.smithi055.stdout:ID="rhel"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="fedora"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8.6"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.redhat.com/"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-01-27T00:56:18.676 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-01-27T00:56:18.677 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server
2024-01-27T00:56:18.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T00:56:18.797 INFO:teuthology.dispatcher.supervisor:Running job 7533484
2024-01-27T00:56:18.797 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533484 --name yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi --description fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full} -- /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533484/orig.config.yaml
2024-01-27T00:56:18.801 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533484
2024-01-27T00:56:18.801 INFO:teuthology.dispatcher.supervisor:Job PID: 756995
2024-01-27T00:56:18.801 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-27T00:58:18.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:00:19.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:02:19.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:04:19.376 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:06:19.541 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:08:19.708 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:10:19.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:12:20.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:14:20.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:16:20.340 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:18:20.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:20:20.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:22:20.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:24:21.024 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:26:21.200 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:28:21.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:30:21.516 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:32:21.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:34:21.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:36:22.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:38:22.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:40:22.408 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:42:22.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:44:22.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:46:22.916 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:48:23.093 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:48:23.158 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-27T01:48:23.295 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-27T01:48:23.295 INFO:teuthology.nuke:Checking targets against current locks
2024-01-27T01:48:23.343 DEBUG:teuthology.nuke:shortname: smithi055
2024-01-27T01:48:23.343 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-27T01:48:23.368 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533484', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 00:51:34.109241', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGxUZGPCY8QOl+rtbwq3uEUTQbbSaNtOygLnR+WuU4qyQSQl/DMA8OVsZnNoxNQrIdoscrcY74/LUfiTINsNaqk='}
2024-01-27T01:48:23.421 INFO:teuthology.orchestra.console.smithi055:Power off
2024-01-27T01:48:23.421 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-27T01:48:23.443 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off
2024-01-27T01:48:23.443 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T01:48:23.469 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off
2024-01-27T01:48:23.570 INFO:teuthology.orchestra.console.smithi055:Power off completed
2024-01-27T01:48:23.724 INFO:teuthology.lock.ops:unlocked: smithi055.front.sepia.ceph.com