2023-07-12T14:43:24.726 INFO:root:teuthology version: 0.0.1.dev140+gb53f236
2023-07-12T14:43:24.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T14:43:24.728 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T14:43:24.882 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7faaa3799580>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7faaa3104700>, signals=[15])
2023-07-12T14:43:24.884 INFO:teuthology.lock.ops:Start node 'smithi171.front.sepia.ceph.com' reimaging
2023-07-12T14:43:24.885 INFO:teuthology.lock.ops:Updating [smithi171.front.sepia.ceph.com]: reset os type and version on server
2023-07-12T14:43:24.885 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server
2023-07-12T14:43:24.927 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /host {"name": "smithi171"}
2023-07-12T14:43:25.020 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-07-12T14:43:25.071 DEBUG:teuthology.provision.fog.smithi171:Requesting image smithi_centos_8.stream (ID 3)
2023-07-12T14:43:25.071 DEBUG:teuthology.provision.fog.smithi171:FOG request: PUT /host/276 {"imageID": 3}
2023-07-12T14:43:25.128 INFO:teuthology.provision.fog.smithi171:Scheduling deploy of centos 8.stream
2023-07-12T14:43:25.129 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:43:25.188 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /tasktype {"name": "deploy"}
2023-07-12T14:43:25.232 DEBUG:teuthology.provision.fog.smithi171:FOG request: POST /host/276/task {"taskTypeID": 1}
2023-07-12T14:43:25.293 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:43:25.382 INFO:teuthology.orchestra.console:Power off smithi171
2023-07-12T14:43:25.385 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-12T14:43:25.416 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-12T14:43:25.542 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2023-07-12T14:43:25.643 INFO:teuthology.orchestra.console:Power on smithi171
2023-07-12T14:43:25.643 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-12T14:43:25.665 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-12T14:43:29.696 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-12T14:43:33.808 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-12T14:43:34.066 INFO:teuthology.orchestra.console:Power on for smithi171 completed
2023-07-12T14:43:34.167 INFO:teuthology.provision.fog.smithi171:Waiting for deploy to finish
2023-07-12T14:43:34.168 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:43:49.236 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:44:04.300 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:44:19.364 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:44:34.428 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:44:49.496 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:45:04.568 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:45:19.632 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:45:34.692 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:45:49.752 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:46:04.812 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:46:19.872 DEBUG:teuthology.provision.fog.smithi171:FOG request: GET /task/active None
2023-07-12T14:46:19.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:46:22.992 WARNING:teuthology.provision.fog.smithi171:[Errno None] Unable to connect to port 22 on 172.21.15.171
2023-07-12T14:46:28.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:46:32.080 WARNING:teuthology.provision.fog.smithi171:[Errno None] Unable to connect to port 22 on 172.21.15.171
2023-07-12T14:46:38.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:46:41.168 WARNING:teuthology.provision.fog.smithi171:[Errno None] Unable to connect to port 22 on 172.21.15.171
2023-07-12T14:46:47.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:46:50.256 WARNING:teuthology.provision.fog.smithi171:[Errno None] Unable to connect to port 22 on 172.21.15.171
2023-07-12T14:46:56.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:46:59.344 WARNING:teuthology.provision.fog.smithi171:[Errno None] Unable to connect to port 22 on 172.21.15.171
2023-07-12T14:47:05.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:47:08.432 WARNING:teuthology.provision.fog.smithi171:[Errno None] Unable to connect to port 22 on 172.21.15.171
2023-07-12T14:47:14.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:47:17.520 WARNING:teuthology.provision.fog.smithi171:[Errno None] Unable to connect to port 22 on 172.21.15.171
2023-07-12T14:47:23.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:48:23.582 WARNING:teuthology.provision.fog.smithi171:timed out
2023-07-12T14:48:29.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:48:29.809 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-12T14:48:29.810 DEBUG:teuthology.orchestra.run.smithi171:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-12T14:48:30.045 INFO:teuthology.provision.fog.smithi171:Node is ready
2023-07-12T14:48:30.045 DEBUG:teuthology.orchestra.run.smithi171:> hostname
2023-07-12T14:48:30.064 INFO:teuthology.orchestra.run.smithi171.stdout:smithi171.front.sepia.ceph.com
2023-07-12T14:48:30.064 DEBUG:teuthology.orchestra.run.smithi171:> grep smithi171.front.sepia.ceph.com /etc/hosts
2023-07-12T14:48:30.123 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.15.171 smithi171.front.sepia.ceph.com smithi171
2023-07-12T14:48:30.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname smithi171
2023-07-12T14:48:30.224 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i -e 's/smithi171.front.sepia.ceph.com/smithi171/g' /etc/hosts
2023-07-12T14:48:30.252 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i -e 's/172.21.15.171/172.21.15.171/g' /etc/hosts
2023-07-12T14:48:30.319 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i -e 's/smithi171.front.sepia.ceph.com/smithi171/g' /etc/hostname
2023-07-12T14:48:30.386 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname smithi171
2023-07-12T14:48:30.452 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release
2023-07-12T14:48:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream"
2023-07-12T14:48:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="8"
2023-07-12T14:48:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos"
2023-07-12T14:48:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora"
2023-07-12T14:48:30.511 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="8"
2023-07-12T14:48:30.512 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el8"
2023-07-12T14:48:30.512 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 8"
2023-07-12T14:48:30.512 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31"
2023-07-12T14:48:30.512 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-07-12T14:48:30.512 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/"
2023-07-12T14:48:30.512 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-12T14:48:30.513 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-12T14:48:30.513 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-12T14:48:30.513 INFO:teuthology.provision.fog.smithi171:Deploy complete!
2023-07-12T14:48:30.515 DEBUG:teuthology.parallel:result is None
2023-07-12T14:48:30.636 INFO:teuthology.lock.ops:Checking smithi171.front.sepia.ceph.com
2023-07-12T14:48:30.636 INFO:teuthology.lock.ops:New key found. Updating...
2023-07-12T14:48:30.662 INFO:teuthology.lock.ops:Updating [smithi171.front.sepia.ceph.com]: set os type and version on server
2023-07-12T14:48:30.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com'
2023-07-12T14:48:30.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-12T14:48:30.727 DEBUG:teuthology.orchestra.run.smithi171:> true
2023-07-12T14:48:30.794 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com'
2023-07-12T14:48:30.795 DEBUG:teuthology.orchestra.run.smithi171:> uname -m
2023-07-12T14:48:30.852 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64
2023-07-12T14:48:30.852 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release
2023-07-12T14:48:30.909 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream"
2023-07-12T14:48:30.909 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="8"
2023-07-12T14:48:30.909 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos"
2023-07-12T14:48:30.909 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora"
2023-07-12T14:48:30.910 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="8"
2023-07-12T14:48:30.910 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el8"
2023-07-12T14:48:30.910 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 8"
2023-07-12T14:48:30.910 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31"
2023-07-12T14:48:30.910 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-07-12T14:48:30.910 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/"
2023-07-12T14:48:30.910 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-12T14:48:30.911 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-12T14:48:30.911 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-12T14:48:30.911 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server
2023-07-12T14:48:30.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T14:48:30.936 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T14:48:31.027 INFO:teuthology.dispatcher.supervisor:Running job 7335360
2023-07-12T14:48:31.028 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology -v --owner scheduled_jcollin@teuthology --archive /home/teuthworker/archive/jcollin-2023-07-12_14:31:50-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7335360 --name jcollin-2023-07-12_14:31:50-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi --description fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} -- /home/teuthworker/archive/jcollin-2023-07-12_14:31:50-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7335360/orig.config.yaml
2023-07-12T14:48:31.031 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/jcollin-2023-07-12_14:31:50-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7335360
2023-07-12T14:48:31.032 INFO:teuthology.dispatcher.supervisor:Job PID: 3177
2023-07-12T14:48:31.032 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-07-12T14:50:31.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T14:50:31.132 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T14:52:31.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T14:52:31.287 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T14:54:31.448 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T14:54:31.449 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T14:56:31.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T14:56:31.621 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T14:58:31.797 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T14:58:31.798 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T15:00:31.978 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T15:00:31.979 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T15:02:32.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T15:02:32.161 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T15:04:32.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T15:04:32.330 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T15:06:32.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T15:06:32.509 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T15:08:32.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-12T15:08:32.678 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-12T15:08:32.743 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-07-12T15:08:32.841 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-07-12T15:08:32.841 INFO:teuthology.nuke:Checking targets against current locks
2023-07-12T15:08:32.893 DEBUG:teuthology.nuke:shortname: smithi171
2023-07-12T15:08:32.894 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-12T15:08:32.920 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2023-07-12_14:31:50-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7335360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-12 14:43:23.817155', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLScnOEuyQCsVtX0HIEaiV6b+WtxLwlKcMgyLbs22AUXu38AXihJfLcC61jnU046aKtfeATQ8/EZkAcrR+rbYaM='}
2023-07-12T15:08:32.973 INFO:teuthology.orchestra.console:Power off smithi171
2023-07-12T15:08:32.974 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-12T15:08:32.995 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-12T15:08:33.116 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2023-07-12T15:08:33.275 INFO:teuthology.lock.ops:unlocked: smithi171.front.sepia.ceph.com