2023-02-11T03:22:16.451 INFO:root:teuthology version: 0.0.1.dev37+g00690ec 2023-02-11T03:22:16.767 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging 2023-02-11T03:22:17.070 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server 2023-02-11T03:22:17.070 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2023-02-11T03:22:17.088 INFO:teuthology.lock.ops:Node 'smithi002.front.sepia.ceph.com' reimaging is complete 2023-02-11T03:22:17.273 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging 2023-02-11T03:22:17.273 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server 2023-02-11T03:22:17.273 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-02-11T03:22:17.291 INFO:teuthology.lock.ops:Node 'smithi050.front.sepia.ceph.com' reimaging is complete 2023-02-11T03:22:17.792 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 8.stream 2023-02-11T03:22:17.876 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 8.stream 2023-02-11T03:22:18.133 INFO:teuthology.orchestra.console:Power off smithi002 2023-02-11T03:22:18.180 INFO:teuthology.orchestra.console:Power off smithi050 2023-02-11T03:22:26.632 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2023-02-11T03:22:26.666 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2023-02-11T03:22:26.733 INFO:teuthology.orchestra.console:Power on smithi002 2023-02-11T03:22:26.767 INFO:teuthology.orchestra.console:Power on smithi050 2023-02-11T03:22:35.111 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2023-02-11T03:22:35.135 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2023-02-11T03:22:35.213 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2023-02-11T03:22:35.236 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2023-02-11T03:25:24.357 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-02-11T03:25:24.370 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-02-11T03:25:33.449 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-02-11T03:25:33.450 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-02-11T03:25:42.533 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-02-11T03:25:42.533 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-02-11T03:25:51.589 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-02-11T03:25:51.589 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-02-11T03:26:57.638 WARNING:teuthology.provision.fog:timed out 2023-02-11T03:26:57.655 WARNING:teuthology.provision.fog:timed out 2023-02-11T03:27:03.953 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-11T03:27:04.011 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-11T03:27:04.231 INFO:teuthology.provision.fog.smithi002:Node is ready 2023-02-11T03:27:04.250 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2023-02-11T03:27:04.295 INFO:teuthology.provision.fog.smithi050:Node is ready 2023-02-11T03:27:04.309 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2023-02-11T03:27:04.356 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2023-02-11T03:27:04.415 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2023-02-11T03:27:04.665 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2023-02-11T03:27:04.791 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2023-02-11T03:27:05.001 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2023-02-11T03:27:05.002 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-11T03:27:05.078 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2023-02-11T03:27:05.078 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-11T03:27:05.094 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2023-02-11T03:27:05.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-02-11T03:27:05.286 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8" 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8" 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 8" 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2023-02-11T03:27:05.346 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-02-11T03:27:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2023-02-11T03:27:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-11T03:27:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-11T03:27:05.347 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-02-11T03:27:05.348 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2023-02-11T03:27:05.364 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2023-02-11T03:27:05.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-02-11T03:27:05.514 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2023-02-11T03:27:05.573 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2023-02-11T03:27:05.573 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8" 2023-02-11T03:27:05.573 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2023-02-11T03:27:05.573 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2023-02-11T03:27:05.573 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8" 2023-02-11T03:27:05.573 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8" 2023-02-11T03:27:05.573 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 8" 2023-02-11T03:27:05.574 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2023-02-11T03:27:05.574 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-02-11T03:27:05.574 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2023-02-11T03:27:05.574 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-11T03:27:05.574 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-11T03:27:05.574 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-02-11T03:27:05.574 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-02-11T03:27:05.663 INFO:teuthology.dispatcher.supervisor:Running job 7160110 2023-02-11T03:27:05.664 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_00690eca1aa0bd56527bb28e9f9e35d734d406bf/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160110 --name yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect} -- /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160110/orig.config.yaml 2023-02-11T03:27:05.696 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160110 2023-02-11T03:27:05.697 INFO:teuthology.dispatcher.supervisor:Job PID: 31236 2023-02-11T03:27:05.697 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-02-11T15:28:11.365 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-02-11T15:28:17.010 INFO:teuthology.kill:Killing Pids: {31236} 2023-02-11T15:28:18.713 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-02-11T15:28:18.713 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-02-11T15:28:18.719 INFO:teuthology.misc:Compressing logs... 2023-02-11T15:28:18.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-02-11T15:28:22.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-02-11T15:28:23.307 INFO:teuthology.kill:No teuthology processes running 2023-02-11T15:28:24.651 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi050'] 2023-02-11T15:28:25.239 INFO:teuthology.kill:2023-02-11 15:28:25,238.238 INFO:teuthology.nuke:targets: 2023-02-11T15:28:25.281 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/kocwrN/tsZCwJFPaOVNz82+Y0jIinJJ4GkD+ocDJgUPndGgtt0kwsrHMJoy8m8mw6CYvsZ9AoDuyfcuKC4zpsX0UElj+U2rKgl58mMX/9WQcQ3+/EnHrpVq6Y9AKD1HmrmptSfp4ITR5OQXHANoqH1JcOQCAXn64bjyWXpXJVZSYtPnFsrEFTfspA0qbc/zsCkNvU9s3LPbkPVJQxVSZNGGDNToXLMEoLerdO4mL3/2wxsvKQzqeNEtluvq4X9v3kQb1pCqkeLOxqTzJRZ3bZdQbN6ZH+HCNFQlBO0vGFkXNM5ATb6fchid1cxa9h/ThXAPGhgzz4OhH3MdKK61GlP+OB1fd7I0MuQnjxusenJsk8BhzlXB/QnI5EifY7alUBwiNTd2bqRrIHdzpSBI1YFZPgSlYfG3km5I4HVFy6DQZWrRCcqQ0ifQl37A5RzwI3UGIhmBrw+Rc710S9wH4RPLudZJ9qi/wA21keUnxGkZMW17vRh4UDAUq8+N0XyU= 2023-02-11T15:28:25.284 INFO:teuthology.kill: smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbAodkvA1Z+VBFAfmvk8RKp2SZUHtdPVDV2oQMZht4HmFnnRLHxmR5ngKxb4L3bAULFm+58Ih/WkO8d+IdhclWuvprt0dWOj2Mx/9A+AUQpjag/l6Aq1JmjCFTojn2Wa92lTOYaLH1QZzayPyn2tsBkN6kGZQevKXhnOExNpX9Xh1NuBvP7vWbDCWWCMggQhJLGGxkqnpiW67UhmFIEYx67tB0IJkzOzlLBfb73CkKuOKz93yUbMOmDrUIPVZfh8noz8Y4I0Z128CBJDfE/PdWUrboO4ta07Q7pvgAspQZE7DEdc+fe27J9OmPUnlPRcsbt2MT7RDUX5yA1RzLC8ueVveNlAhd80ucw3wV2IBm0AhIqZisWM/n4kBFsWyv4BDnaCpGWeNQlVH8vU0bLlvFlw2DPHPYq1U9NFCbG4TizLsPJyLgu5QQlRY2EEVxAvGOuYEt6yPkau1tdnC+QPDw7Dk1+gG0vxVlb2m8eNQgwi22obMoqPB7o1wV4v5MhnE= 2023-02-11T15:28:25.285 INFO:teuthology.kill:2023-02-11 15:28:25,239.239 INFO:teuthology.nuke:Checking targets against current locks 2023-02-11T15:28:25.285 INFO:teuthology.kill:2023-02-11 15:28:25,284.284 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-11T15:28:25.310 INFO:teuthology.kill:2023-02-11 15:28:25,310.310 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-11T15:28:25.362 INFO:teuthology.kill:2023-02-11 15:28:25,361.361 INFO:teuthology.orchestra.console:Power off smithi002 2023-02-11T15:28:25.377 INFO:teuthology.kill:2023-02-11 15:28:25,377.377 INFO:teuthology.orchestra.console:Power off smithi050 2023-02-11T15:28:33.742 INFO:teuthology.kill:2023-02-11 15:28:33,741.741 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2023-02-11T15:28:33.749 INFO:teuthology.kill:2023-02-11 15:28:33,749.749 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2023-02-11T15:28:33.898 INFO:teuthology.kill:2023-02-11 15:28:33,897.897 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com 2023-02-11T15:28:33.902 INFO:teuthology.kill:2023-02-11 15:28:33,902.902 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com 2023-02-11T15:30:34.329 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15