2021-09-23T05:25:46.712 INFO:root:teuthology version: 1.1.0-d3a488f5 2021-09-23T05:25:46.833 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi026.front.sepia.ceph.com' reimaging 2021-09-23T05:25:46.835 INFO:teuthology.lock.ops:Updating [ubuntu@smithi026.front.sepia.ceph.com]: reset os type and version on server 2021-09-23T05:25:46.835 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2021-09-23T05:25:46.850 INFO:teuthology.lock.ops:Node 'ubuntu@smithi026.front.sepia.ceph.com' reimaging is complete 2021-09-23T05:25:46.851 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging 2021-09-23T05:25:46.851 INFO:teuthology.lock.ops:Updating [ubuntu@smithi059.front.sepia.ceph.com]: reset os type and version on server 2021-09-23T05:25:46.851 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2021-09-23T05:25:46.866 INFO:teuthology.lock.ops:Node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging is complete 2021-09-23T05:25:46.867 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi131.front.sepia.ceph.com' reimaging 2021-09-23T05:25:46.867 INFO:teuthology.lock.ops:Updating [ubuntu@smithi131.front.sepia.ceph.com]: reset os type and version on server 2021-09-23T05:25:46.867 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2021-09-23T05:25:46.883 INFO:teuthology.lock.ops:Node 'ubuntu@smithi131.front.sepia.ceph.com' reimaging is complete 2021-09-23T05:25:47.068 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of centos 8.stream 2021-09-23T05:25:47.072 INFO:teuthology.provision.fog.smithi026:Scheduling deploy of centos 8.stream 2021-09-23T05:25:47.101 INFO:teuthology.provision.fog.smithi131:Scheduling deploy of centos 8.stream 2021-09-23T05:25:47.475 INFO:teuthology.orchestra.console:Power off smithi059 2021-09-23T05:25:47.534 INFO:teuthology.orchestra.console:Power off smithi026 2021-09-23T05:25:47.569 INFO:teuthology.orchestra.console:Power off smithi131 2021-09-23T05:25:55.900 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2021-09-23T05:25:55.939 INFO:teuthology.orchestra.console:Power off for smithi131 completed 2021-09-23T05:25:56.002 INFO:teuthology.orchestra.console:Power on smithi026 2021-09-23T05:25:56.040 INFO:teuthology.orchestra.console:Power on smithi131 2021-09-23T05:25:59.955 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2021-09-23T05:26:00.057 INFO:teuthology.orchestra.console:Power on smithi059 2021-09-23T05:26:04.310 INFO:teuthology.orchestra.console:Power on for smithi059 completed 2021-09-23T05:26:04.370 INFO:teuthology.orchestra.console:Power on for smithi026 completed 2021-09-23T05:26:04.411 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2021-09-23T05:26:04.417 INFO:teuthology.orchestra.console:Power on for smithi131 completed 2021-09-23T05:26:04.471 INFO:teuthology.provision.fog.smithi026:Waiting for deploy to finish 2021-09-23T05:26:04.519 INFO:teuthology.provision.fog.smithi131:Waiting for deploy to finish 2021-09-23T05:29:56.141 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-23T05:30:12.302 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-23T05:30:12.555 INFO:teuthology.provision.fog.smithi026:Node is ready 2021-09-23T05:30:12.612 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026.front.sepia.ceph.com 2021-09-23T05:30:12.669 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26 smithi026.front.sepia.ceph.com smithi026 2021-09-23T05:30:13.023 INFO:teuthology.provision.fog.smithi026:Deploy complete! 2021-09-23T05:30:16.476 INFO:teuthology.provision.fog.smithi131:Node is ready 2021-09-23T05:30:16.497 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131.front.sepia.ceph.com 2021-09-23T05:30:16.554 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131 smithi131.front.sepia.ceph.com smithi131 2021-09-23T05:30:16.913 INFO:teuthology.provision.fog.smithi131:Deploy complete! 2021-09-23T05:30:21.251 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-23T05:30:21.503 INFO:teuthology.provision.fog.smithi059:Node is ready 2021-09-23T05:30:21.526 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2021-09-23T05:30:21.585 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2021-09-23T05:30:21.936 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2021-09-23T05:30:22.098 INFO:teuthology.lock.ops:Checking smithi026.front.sepia.ceph.com 2021-09-23T05:30:22.100 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-23T05:30:22.118 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2021-09-23T05:30:22.119 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-23T05:30:22.134 INFO:teuthology.lock.ops:Checking smithi131.front.sepia.ceph.com 2021-09-23T05:30:22.135 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-23T05:30:22.149 INFO:teuthology.lock.ops:Updating [smithi026.front.sepia.ceph.com]: set os type and version on server 2021-09-23T05:30:22.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2021-09-23T05:30:22.394 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2021-09-23T05:30:22.456 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2021-09-23T05:30:22.458 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8" 2021-09-23T05:30:22.461 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2021-09-23T05:30:22.462 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2021-09-23T05:30:22.462 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8" 2021-09-23T05:30:22.463 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2021-09-23T05:30:22.465 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 8" 2021-09-23T05:30:22.465 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2021-09-23T05:30:22.465 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-09-23T05:30:22.466 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2021-09-23T05:30:22.467 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-09-23T05:30:22.467 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-09-23T05:30:22.467 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-09-23T05:30:22.470 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2021-09-23T05:30:22.487 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2021-09-23T05:30:22.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2021-09-23T05:30:22.718 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2021-09-23T05:30:22.774 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2021-09-23T05:30:22.775 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8" 2021-09-23T05:30:22.775 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2021-09-23T05:30:22.775 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2021-09-23T05:30:22.776 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8" 2021-09-23T05:30:22.776 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2021-09-23T05:30:22.776 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 8" 2021-09-23T05:30:22.777 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2021-09-23T05:30:22.777 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-09-23T05:30:22.777 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2021-09-23T05:30:22.778 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-09-23T05:30:22.778 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-09-23T05:30:22.779 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-09-23T05:30:22.779 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2021-09-23T05:30:22.797 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: set os type and version on server 2021-09-23T05:30:22.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2021-09-23T05:30:22.998 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2021-09-23T05:30:23.055 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2021-09-23T05:30:23.055 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8" 2021-09-23T05:30:23.056 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2021-09-23T05:30:23.056 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2021-09-23T05:30:23.056 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8" 2021-09-23T05:30:23.056 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2021-09-23T05:30:23.057 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 8" 2021-09-23T05:30:23.057 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2021-09-23T05:30:23.057 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-09-23T05:30:23.057 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2021-09-23T05:30:23.058 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-09-23T05:30:23.058 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-09-23T05:30:23.058 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-09-23T05:30:23.059 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2021-09-23T05:30:23.145 INFO:teuthology.dispatcher.supervisor:Running job 6403511 2021-09-23T05:30:23.146 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d3a488f52e40d4e803797684b740cd06ea526c40/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-09-23_01:04:10-rados-wip-yuri6-testing-2021-09-22-1530-distro-basic-smithi/6403511 --name yuriw-2021-09-23_01:04:10-rados-wip-yuri6-testing-2021-09-22-1530-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/yuriw-2021-09-23_01:04:10-rados-wip-yuri6-testing-2021-09-22-1530-distro-basic-smithi/6403511/orig.config.yaml 2021-09-23T05:30:23.149 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-09-23_01:04:10-rados-wip-yuri6-testing-2021-09-22-1530-distro-basic-smithi/6403511 2021-09-23T05:30:23.150 INFO:teuthology.dispatcher.supervisor:Job PID: 23929 2021-09-23T05:30:23.151 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-09-23T17:31:12.847 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-09-23T17:31:13.100 INFO:teuthology.kill:Killing Pids: {23929} 2021-09-23T17:31:14.150 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2021-09-23T17:31:14.151 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2021-09-23T17:31:14.151 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2021-09-23T17:31:14.151 INFO:teuthology.misc:Compressing logs... 2021-09-23T17:31:14.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2021-09-23T17:31:14.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2021-09-23T17:31:14.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2021-09-23T17:31:15.216 INFO:teuthology.misc:Compressing logs... 2021-09-23T17:31:15.895 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-09-23T17:31:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-09-23T17:31:16.134 INFO:teuthology.orchestra.run.smithi026.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-09-23T17:31:19.357 INFO:teuthology.misc:Compressing logs... 2021-09-23T17:33:13.240 INFO:teuthology.kill:No teuthology processes running 2021-09-23T17:33:14.246 INFO:teuthology.kill:Nuking machines: ['smithi026', 'smithi059', 'smithi131'] 2021-09-23T17:33:15.093 INFO:teuthology.kill:2021-09-23 17:33:15,093.093 INFO:teuthology.nuke:targets: 2021-09-23T17:33:15.094 INFO:teuthology.kill: smithi026.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ1lAPsllD121T/CpKFSH/pGZnmwJ59j4/eZ4ZH3Msy+kF7ukjSmWeztW104ZPznei/h5tmUMcgXY7GTciET7i7vusV3lzGFrK/okYgDjS7+Tgq6gJZWEi9lFw7CVr96AcdSiYQvAXIa4ODJQ02/Pt5pHiBbuyE7k+f+vcTMjXfPKRRW20njZ/qO4veaZb72jd8b9r9bgCidGhgT6XUnrCRBycnVH4ytEj0xBzkJyc7Z5KFxPH1K+ljY6gbNN5e97G6Q+rYdfh2IDMSKCwW1/sw7b9cCA3abE7CfBkcm1SmuDOLsT2PPNbYldHwEFhpit2FYfbt+sFxxqOesdwh61akKtUcSndPjORAbTDc2CZYGWpW3fIvFO9Od+Mz38oHinO1n1zCxi+h0H4gKNYV06adNHQDnpUhnPVNBYsrb5nGsTg3WPV6PcbCm/2tENoeZZpLhkZSX9dUa5iFqedA/5yabHjJz1fixe6MOpoz0t+sP0WpZC5aOytfBViLugxXGM= 2021-09-23T17:33:15.094 INFO:teuthology.kill: smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Z/ymAEI8LXZYVzRXOmN9WrLadjk+iirXkI/vYXvwl8gZCjy+O9qJWYGBYDReirSvXSTNqnULbSP5yKlRgqLy9WIO3w0aGuBnms2vFMvNmZ6rqoDAVSmqEc4G8p/eKVAzUWPl9RvO9idKE82CcBJdcfVeSx6flhfK71WsDYUMcp1kDtCY+HXnMTG7nLRFPP99LueumvLtd4anuJatE8Xt0YUdMN6V+Pm9bohEolQSpnT9NQZ9TZFoqAzz70rwQPWq27hyHLUURwcud/Mu7xgdnGJtWygZxqYiJs5Za26ID29JO/JZMBz5wf1d/uuhayw59jbc71BncHR5XixIEYh6Z2kyZU8qo5HuwQoVEC9yB8PAvgcDyWQKUgRhpBH93F2p0MyEiYajGxzQqw2ZEEAyGDKAPsYJv/ThEebXDNYOInHiSwhbYbpo+LkI+AxWI+lSrcSTreF5KGJ2dK/sYOhkt79ueb3+a5E8J35ZirOYODirWmHPZ3TFFb7DWfMDVtU= 2021-09-23T17:33:15.094 INFO:teuthology.kill: smithi131.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcpU9xJgUObV6kWxx3wNJHzyb/b5wzBjUj4ogw2Szs3qvDFcbS3z57DJ3V6qRBtIJlfC/tFvEyG7wz+RKwCtghGWwDaCUn2hcqVsyUhn486zcl+7twbcH0z/ZbnMA991YmXPwqwVyzVrTBd/XOUZoO9YQ1Uq3WdkAI76z3PvjmcruKLUV8Ladq1BV52cnXCMmuAKrW7z+zbCwSYMWeH6newA6/osQ95576k/h3hoqNtFol0nPrAdvFQIAeS/O2gaC0UIpbI7i/VEQOCClxizQPNma9PEFeFNxgC4rN0yrWRBQq3f0+X2uVM3bMm9qjrKDOUp/FY7N0Af08Lj8I4Q1lI4Oels05TmYRnwdFTlo3To3AMqOnLN6cxPvGRPMrUUKPQPJsuIkr6swGZOGU3lowY4iiwSP4M6PL8D9lU/FJa8WdDWGBKHUuwRndwCmWXB2kmanXVjZf0CqyK7/bm923Gov39RZ4CsMKM2+/obSCi8CgkTfzt165Py4YBk6lD6s= 2021-09-23T17:33:15.114 INFO:teuthology.kill:2021-09-23 17:33:15,114.114 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-23T17:33:15.117 INFO:teuthology.kill:2021-09-23 17:33:15,117.117 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-23T17:33:15.120 INFO:teuthology.kill:2021-09-23 17:33:15,119.119 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-23T17:33:15.173 INFO:teuthology.kill:2021-09-23 17:33:15,173.173 INFO:teuthology.orchestra.console:Power off smithi131 2021-09-23T17:33:15.185 INFO:teuthology.kill:2021-09-23 17:33:15,185.185 INFO:teuthology.orchestra.console:Power off smithi026 2021-09-23T17:33:15.196 INFO:teuthology.kill:2021-09-23 17:33:15,195.195 INFO:teuthology.orchestra.console:Power off smithi059 2021-09-23T17:33:23.555 INFO:teuthology.kill:2021-09-23 17:33:23,555.555 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2021-09-23T17:33:23.561 INFO:teuthology.kill:2021-09-23 17:33:23,560.560 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2021-09-23T17:33:23.696 INFO:teuthology.kill:2021-09-23 17:33:23,695.695 INFO:teuthology.lock.ops:unlocked: smithi026.front.sepia.ceph.com 2021-09-23T17:33:23.700 INFO:teuthology.kill:2021-09-23 17:33:23,700.700 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2021-09-23T17:33:27.661 INFO:teuthology.kill:2021-09-23 17:33:27,661.661 INFO:teuthology.orchestra.console:Power off for smithi131 completed 2021-09-23T17:33:27.799 INFO:teuthology.kill:2021-09-23 17:33:27,799.799 INFO:teuthology.lock.ops:unlocked: smithi131.front.sepia.ceph.com 2021-09-23T17:35:28.008 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15