2023-09-12T02:06:44.103 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-12T02:06:44.104 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:06:44.214 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fba59b2ea60>, signals=[15]) 2023-09-12T02:06:44.215 INFO:teuthology.lock.ops:Start node 'smithi160.front.sepia.ceph.com' reimaging 2023-09-12T02:06:44.216 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: reset os type and version on server 2023-09-12T02:06:44.216 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2023-09-12T02:06:44.236 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /host {"name": "smithi160"} 2023-09-12T02:06:44.310 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-12T02:06:44.347 DEBUG:teuthology.provision.fog.smithi160:Requesting image smithi_centos_8.stream (ID 3) 2023-09-12T02:06:44.347 DEBUG:teuthology.provision.fog.smithi160:FOG request: PUT /host/265 {"imageID": 3} 2023-09-12T02:06:44.391 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of centos 8.stream 2023-09-12T02:06:44.391 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:06:44.466 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /tasktype {"name": "deploy"} 2023-09-12T02:06:44.507 DEBUG:teuthology.provision.fog.smithi160:FOG request: POST /host/265/task {"taskTypeID": 1} 2023-09-12T02:06:44.566 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:06:44.669 INFO:teuthology.orchestra.console.smithi160:Power off 2023-09-12T02:06:44.670 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-12T02:06:44.690 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off 2023-09-12T02:06:44.691 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T02:06:44.713 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2023-09-12T02:06:44.814 INFO:teuthology.orchestra.console.smithi160:Power off completed 2023-09-12T02:06:44.915 INFO:teuthology.orchestra.console.smithi160:Power on 2023-09-12T02:06:44.915 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-12T02:06:44.936 DEBUG:teuthology.orchestra.console.smithi160:power on output: Chassis Power Control: Up/On 2023-09-12T02:06:44.937 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T02:06:44.958 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2023-09-12T02:06:48.962 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T02:06:49.070 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2023-09-12T02:06:53.074 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T02:06:53.183 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2023-09-12T02:06:53.284 INFO:teuthology.orchestra.console.smithi160:Power on completed 2023-09-12T02:06:53.384 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish 2023-09-12T02:06:53.385 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:07:08.520 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:07:23.660 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:07:38.802 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:07:53.925 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:08:09.065 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:08:24.209 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:08:39.343 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:08:54.533 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:09:09.700 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:09:24.838 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:09:39.975 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:09:55.122 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:10:10.214 DEBUG:teuthology.provision.fog.smithi160:FOG request: GET /task/active None 2023-09-12T02:10:10.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:10:28.634 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-09-12T02:10:34.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:10:37.722 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-09-12T02:10:43.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:10:46.778 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-09-12T02:10:52.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:10:55.834 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-09-12T02:11:01.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:11:04.890 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-09-12T02:11:10.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:11:13.946 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-09-12T02:11:19.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:11:38.522 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-09-12T02:11:44.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:11:47.610 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-09-12T02:11:53.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:11:54.222 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-12T02:11:54.223 DEBUG:teuthology.orchestra.run.smithi160:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-12T02:12:19.788 INFO:teuthology.provision.fog.smithi160:Node is ready 2023-09-12T02:12:19.789 DEBUG:teuthology.orchestra.run.smithi160:> hostname 2023-09-12T02:12:19.813 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com 2023-09-12T02:12:19.813 DEBUG:teuthology.orchestra.run.smithi160:> grep smithi160.front.sepia.ceph.com /etc/hosts 2023-09-12T02:12:19.873 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160 2023-09-12T02:12:19.874 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2023-09-12T02:12:19.994 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hosts 2023-09-12T02:12:20.025 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/172.21.15.160/172.21.15.160/g' /etc/hosts 2023-09-12T02:12:20.094 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hostname 2023-09-12T02:12:20.163 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2023-09-12T02:12:20.229 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2023-09-12T02:12:20.287 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2023-09-12T02:12:20.287 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8" 2023-09-12T02:12:20.288 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2023-09-12T02:12:20.288 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2023-09-12T02:12:20.288 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8" 2023-09-12T02:12:20.288 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2023-09-12T02:12:20.288 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-12T02:12:20.289 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2023-09-12T02:12:20.289 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-12T02:12:20.289 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2023-09-12T02:12:20.289 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-12T02:12:20.290 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-12T02:12:20.290 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-12T02:12:20.291 INFO:teuthology.provision.fog.smithi160:Deploy complete! 2023-09-12T02:12:20.292 DEBUG:teuthology.parallel:result is None 2023-09-12T02:12:20.402 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com 2023-09-12T02:12:20.403 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-12T02:12:20.423 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server 2023-09-12T02:12:20.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-09-12T02:12:20.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T02:12:20.489 DEBUG:teuthology.orchestra.run.smithi160:> true 2023-09-12T02:12:20.555 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-09-12T02:12:20.556 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2023-09-12T02:12:20.613 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2023-09-12T02:12:20.614 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2023-09-12T02:12:20.671 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2023-09-12T02:12:20.671 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8" 2023-09-12T02:12:20.672 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2023-09-12T02:12:20.672 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2023-09-12T02:12:20.672 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8" 2023-09-12T02:12:20.672 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2023-09-12T02:12:20.672 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-12T02:12:20.673 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2023-09-12T02:12:20.673 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-12T02:12:20.673 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2023-09-12T02:12:20.673 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-12T02:12:20.673 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-12T02:12:20.674 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-12T02:12:20.674 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2023-09-12T02:12:20.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:12:20.784 INFO:teuthology.dispatcher.supervisor:Running job 7393824 2023-09-12T02:12:20.785 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_svelar@teuthology --archive /home/teuthworker/archive/svelar-2023-09-12_02:05:33-crimson-rados-wip-svelar-m-distro-default-smithi/7393824 --name svelar-2023-09-12_02:05:33-crimson-rados-wip-svelar-m-distro-default-smithi --description crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_write} -- /home/teuthworker/archive/svelar-2023-09-12_02:05:33-crimson-rados-wip-svelar-m-distro-default-smithi/7393824/orig.config.yaml 2023-09-12T02:12:20.789 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/svelar-2023-09-12_02:05:33-crimson-rados-wip-svelar-m-distro-default-smithi/7393824 2023-09-12T02:12:20.790 INFO:teuthology.dispatcher.supervisor:Job PID: 3168150 2023-09-12T02:12:20.790 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-12T02:14:20.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:16:21.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:18:21.226 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:20:21.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:22:21.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:24:21.726 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:26:21.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:28:22.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:30:22.227 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:32:22.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:34:22.552 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:36:22.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:38:22.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:40:23.058 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T02:40:23.124 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-09-12T02:40:23.207 INFO:teuthology.dispatcher.supervisor:Unlocking machines... 2023-09-12T02:40:23.258 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com