2023-06-23T02:18:20.483 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb
2023-06-23T02:18:20.485 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:18:20.485 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:18:20.613 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7ff11cf4c700>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7ff11e1688b0>, signals=[15])
2023-06-23T02:18:20.615 INFO:teuthology.lock.ops:Start node 'smithi158.front.sepia.ceph.com' reimaging
2023-06-23T02:18:20.615 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: reset os type and version on server
2023-06-23T02:18:20.616 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2023-06-23T02:18:20.646 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /host {"name": "smithi158"}
2023-06-23T02:18:20.722 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-06-23T02:18:20.766 DEBUG:teuthology.provision.fog.smithi158:Requesting image smithi_rhel_8.4 (ID 7)
2023-06-23T02:18:20.767 DEBUG:teuthology.provision.fog.smithi158:FOG request: PUT /host/263 {"imageID": 7}
2023-06-23T02:18:20.823 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of rhel 8.4
2023-06-23T02:18:20.823 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:18:20.901 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /tasktype {"name": "deploy"}
2023-06-23T02:18:20.940 DEBUG:teuthology.provision.fog.smithi158:FOG request: POST /host/263/task {"taskTypeID": 1}
2023-06-23T02:18:20.994 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:18:21.106 INFO:teuthology.orchestra.console:Power off smithi158
2023-06-23T02:18:21.107 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-23T02:18:21.134 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-23T02:18:25.163 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-23T02:18:29.275 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-23T02:18:29.486 INFO:teuthology.orchestra.console:Power off for smithi158 completed
2023-06-23T02:18:29.587 INFO:teuthology.orchestra.console:Power on smithi158
2023-06-23T02:18:29.587 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-23T02:18:29.608 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-23T02:18:33.632 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-23T02:18:37.744 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-23T02:18:37.954 INFO:teuthology.orchestra.console:Power on for smithi158 completed
2023-06-23T02:18:38.055 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish
2023-06-23T02:18:38.056 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:18:53.136 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:19:08.232 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:19:23.312 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:19:38.400 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:19:53.484 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:20:08.564 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:20:23.668 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:20:38.748 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:20:53.836 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:21:08.920 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:21:23.996 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:21:39.063 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-06-23T02:21:39.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-23T02:21:39.412 ERROR:teuthology.orchestra.connection:Error authenticating with smithi158.front.sepia.ceph.com: Authentication failed.
2023-06-23T02:22:40.416 WARNING:teuthology.provision.fog.smithi158:timed out
2023-06-23T02:22:46.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-23T02:22:49.492 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-06-23T02:22:55.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-23T02:22:58.544 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-06-23T02:23:04.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-23T02:24:04.547 WARNING:teuthology.provision.fog.smithi158:timed out
2023-06-23T02:24:10.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-23T02:24:10.912 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-23T02:24:10.913 DEBUG:teuthology.orchestra.run.smithi158:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-23T02:24:11.216 INFO:teuthology.provision.fog.smithi158:Node is ready
2023-06-23T02:24:11.217 DEBUG:teuthology.orchestra.run.smithi158:> hostname
2023-06-23T02:24:11.233 INFO:teuthology.orchestra.run.smithi158.stdout:smithi158.front.sepia.ceph.com
2023-06-23T02:24:11.234 DEBUG:teuthology.orchestra.run.smithi158:> grep smithi158.front.sepia.ceph.com /etc/hosts
2023-06-23T02:24:11.288 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.158 smithi158.front.sepia.ceph.com smithi158
2023-06-23T02:24:11.289 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158
2023-06-23T02:24:11.402 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hosts
2023-06-23T02:24:11.427 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/172.21.15.158/172.21.15.158/g' /etc/hosts
2023-06-23T02:24:11.494 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hostname
2023-06-23T02:24:11.560 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158
2023-06-23T02:24:11.623 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release
2023-06-23T02:24:11.677 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="Red Hat Enterprise Linux"
2023-06-23T02:24:11.677 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8.4 (Ootpa)"
2023-06-23T02:24:11.677 INFO:teuthology.orchestra.run.smithi158.stdout:ID="rhel"
2023-06-23T02:24:11.678 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="fedora"
2023-06-23T02:24:11.678 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8.4"
2023-06-23T02:24:11.678 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8"
2023-06-23T02:24:11.678 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-23T02:24:11.678 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31"
2023-06-23T02:24:11.678 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-23T02:24:11.678 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://www.redhat.com/"
2023-06-23T02:24:11.679 INFO:teuthology.orchestra.run.smithi158.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-23T02:24:11.679 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-23T02:24:11.679 INFO:teuthology.orchestra.run.smithi158.stdout:
2023-06-23T02:24:11.679 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-23T02:24:11.679 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-23T02:24:11.679 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-23T02:24:11.679 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-23T02:24:11.680 INFO:teuthology.provision.fog.smithi158:Deploy complete!
2023-06-23T02:24:11.682 DEBUG:teuthology.parallel:result is None
2023-06-23T02:24:11.809 INFO:teuthology.lock.ops:Checking smithi158.front.sepia.ceph.com
2023-06-23T02:24:11.810 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-23T02:24:11.846 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: set os type and version on server
2023-06-23T02:24:11.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-06-23T02:24:11.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-23T02:24:11.912 DEBUG:teuthology.orchestra.run.smithi158:> true
2023-06-23T02:24:11.979 DEBUG:teuthology.orchestra.run.smithi158:> uname -m
2023-06-23T02:24:12.032 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64
2023-06-23T02:24:12.033 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release
2023-06-23T02:24:12.087 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="Red Hat Enterprise Linux"
2023-06-23T02:24:12.087 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8.4 (Ootpa)"
2023-06-23T02:24:12.087 INFO:teuthology.orchestra.run.smithi158.stdout:ID="rhel"
2023-06-23T02:24:12.088 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="fedora"
2023-06-23T02:24:12.088 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8.4"
2023-06-23T02:24:12.088 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8"
2023-06-23T02:24:12.088 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-23T02:24:12.089 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31"
2023-06-23T02:24:12.089 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-23T02:24:12.089 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://www.redhat.com/"
2023-06-23T02:24:12.089 INFO:teuthology.orchestra.run.smithi158.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-23T02:24:12.089 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-23T02:24:12.089 INFO:teuthology.orchestra.run.smithi158.stdout:
2023-06-23T02:24:12.090 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-23T02:24:12.090 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-23T02:24:12.090 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-23T02:24:12.090 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-23T02:24:12.091 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2023-06-23T02:24:12.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:24:12.120 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:24:12.213 INFO:teuthology.dispatcher.supervisor:Running job 7312529
2023-06-23T02:24:12.213 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-22_20:09:40-rados-wip-yuri6-testing-2023-06-22-1005-quincy-distro-default-smithi/7312529 --name yuriw-2023-06-22_20:09:40-rados-wip-yuri6-testing-2023-06-22-1005-quincy-distro-default-smithi --description rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} -- /home/teuthworker/archive/yuriw-2023-06-22_20:09:40-rados-wip-yuri6-testing-2023-06-22-1005-quincy-distro-default-smithi/7312529/orig.config.yaml
2023-06-23T02:24:12.217 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-06-22_20:09:40-rados-wip-yuri6-testing-2023-06-22-1005-quincy-distro-default-smithi/7312529
2023-06-23T02:24:12.218 INFO:teuthology.dispatcher.supervisor:Job PID: 9255
2023-06-23T02:24:12.218 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-06-23T02:26:12.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:26:12.220 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:28:12.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:28:12.298 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:30:12.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:30:12.380 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:32:12.456 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:32:12.457 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:34:12.536 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:34:12.537 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:36:12.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:36:12.616 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:38:12.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:38:12.686 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:40:12.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:40:12.760 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:42:12.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-23T02:42:12.841 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-23T02:42:12.919 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-06-23T02:42:12.990 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-06-23T02:42:12.990 INFO:teuthology.nuke:Checking targets against current locks
2023-06-23T02:42:13.041 DEBUG:teuthology.nuke:shortname: smithi158
2023-06-23T02:42:13.042 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-23T02:42:13.077 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_20:09:40-rados-wip-yuri6-testing-2023-06-22-1005-quincy-distro-default-smithi/7312529', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 02:18:19.672078', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk5QIWePyfHjiRr5FQULRaSu7BlmhcRqYBOMEkFPlmgSbovmjtozUUj/l5LBJBVbEIrTnAg3wpFuwBnGmCflzdvbz/npnjQnGCyX5sA47nipmc3+TIRiA/56nJiz7TyIJ9P6w2LKmsV4zjXbCyOmz2m9LjQkdo3iMFo1xZUeGzQvlOtZdKzgQKzHvzpdc/KyONRybUbHmM8Mggshwh0oKe6CCH4lftQ7SskXVf7BuwyxJ3MDyE14aa5vchxnLyJoLMvsjz0g/0hBwwnWnp33wYbQfvA/nsmDX4aON/wnnNLZGsN2A4ud0ojkRUse7alEzX3u8D8vwpTBRf8e/aPYUYnN7i94mAJu7VeiRv928cNs7VhQ+DZVkXaa5gvmEmvED+twQbJPEM9RFMntMm6dU6c3DaquFSl8gx5/1uWp2Cow/Cy0vemAuGMjmhp3KJpEclZiR69/AfMIHypMV4SAT7d40cJvG+q0CBfkLDA7z0M+ItqhcvcF1b9KrDX0pG318='}
2023-06-23T02:42:13.135 INFO:teuthology.orchestra.console:Power off smithi158
2023-06-23T02:42:13.136 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-23T02:42:13.180 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-23T02:42:13.302 INFO:teuthology.orchestra.console:Power off for smithi158 completed
2023-06-23T02:42:13.460 INFO:teuthology.lock.ops:unlocked: smithi158.front.sepia.ceph.com