2021-02-23T02:54:50.540 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-23T02:54:50.738 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi080.front.sepia.ceph.com' reimaging 2021-02-23T02:54:50.739 INFO:teuthology.lock.ops:Updating [ubuntu@smithi080.front.sepia.ceph.com]: reset os type and version on server 2021-02-23T02:54:50.739 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2021-02-23T02:54:50.755 INFO:teuthology.lock.ops:Node 'ubuntu@smithi080.front.sepia.ceph.com' reimaging is complete 2021-02-23T02:54:50.756 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi085.front.sepia.ceph.com' reimaging 2021-02-23T02:54:50.756 INFO:teuthology.lock.ops:Updating [ubuntu@smithi085.front.sepia.ceph.com]: reset os type and version on server 2021-02-23T02:54:50.757 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2021-02-23T02:54:50.771 INFO:teuthology.lock.ops:Node 'ubuntu@smithi085.front.sepia.ceph.com' reimaging is complete 2021-02-23T02:54:50.771 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging 2021-02-23T02:54:50.771 INFO:teuthology.lock.ops:Updating [ubuntu@smithi120.front.sepia.ceph.com]: reset os type and version on server 2021-02-23T02:54:50.772 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2021-02-23T02:54:50.787 INFO:teuthology.lock.ops:Node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging is complete 2021-02-23T02:54:50.788 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging 2021-02-23T02:54:50.788 INFO:teuthology.lock.ops:Updating [ubuntu@smithi121.front.sepia.ceph.com]: reset os type and version on server 2021-02-23T02:54:50.788 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2021-02-23T02:54:50.809 INFO:teuthology.lock.ops:Node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging is complete 2021-02-23T02:54:50.809 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging 2021-02-23T02:54:50.810 INFO:teuthology.lock.ops:Updating [ubuntu@smithi172.front.sepia.ceph.com]: reset os type and version on server 2021-02-23T02:54:50.811 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2021-02-23T02:54:50.826 INFO:teuthology.lock.ops:Node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging is complete 2021-02-23T02:54:50.979 INFO:teuthology.provision.fog.smithi080:Scheduling deploy of centos 7.8 2021-02-23T02:54:50.998 INFO:teuthology.provision.fog.smithi085:Scheduling deploy of centos 7.8 2021-02-23T02:54:51.035 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 7.8 2021-02-23T02:54:51.040 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of centos 7.8 2021-02-23T02:54:51.064 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of centos 7.8 2021-02-23T02:54:51.285 INFO:teuthology.orchestra.console:Power off smithi080 2021-02-23T02:54:51.343 INFO:teuthology.orchestra.console:Power off smithi172 2021-02-23T02:54:51.358 INFO:teuthology.orchestra.console:Power off smithi085 2021-02-23T02:54:51.417 INFO:teuthology.orchestra.console:Power off smithi121 2021-02-23T02:54:51.437 INFO:teuthology.orchestra.console:Power off smithi120 2021-02-23T02:54:59.740 INFO:teuthology.orchestra.console:Power off for smithi080 completed 2021-02-23T02:54:59.794 INFO:teuthology.orchestra.console:Power off for smithi085 completed 2021-02-23T02:54:59.832 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2021-02-23T02:54:59.844 INFO:teuthology.orchestra.console:Power on smithi080 2021-02-23T02:54:59.895 INFO:teuthology.orchestra.console:Power on smithi085 2021-02-23T02:54:59.939 INFO:teuthology.orchestra.console:Power on smithi120 2021-02-23T02:55:03.888 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2021-02-23T02:55:03.923 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2021-02-23T02:55:03.990 INFO:teuthology.orchestra.console:Power on smithi172 2021-02-23T02:55:04.026 INFO:teuthology.orchestra.console:Power on smithi121 2021-02-23T02:55:08.239 INFO:teuthology.orchestra.console:Power on for smithi080 completed 2021-02-23T02:55:08.289 INFO:teuthology.orchestra.console:Power on for smithi085 completed 2021-02-23T02:55:08.320 INFO:teuthology.orchestra.console:Power on for smithi120 completed 2021-02-23T02:55:08.341 INFO:teuthology.provision.fog.smithi080:Waiting for deploy to finish 2021-02-23T02:55:08.390 INFO:teuthology.provision.fog.smithi085:Waiting for deploy to finish 2021-02-23T02:55:08.422 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish 2021-02-23T02:55:12.368 INFO:teuthology.orchestra.console:Power on for smithi172 completed 2021-02-23T02:55:12.407 INFO:teuthology.orchestra.console:Power on for smithi121 completed 2021-02-23T02:55:12.469 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2021-02-23T02:55:12.509 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2021-02-23T02:59:39.644 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-23T02:59:40.046 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-23T02:59:40.070 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-23T02:59:40.618 INFO:teuthology.provision.fog.smithi120:Node is ready 2021-02-23T02:59:40.621 INFO:teuthology.provision.fog.smithi085:Node is ready 2021-02-23T02:59:40.702 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com 2021-02-23T02:59:40.706 INFO:teuthology.orchestra.run.smithi085.stdout:smithi085 2021-02-23T02:59:40.775 INFO:teuthology.orchestra.run.smithi085.stdout:172.21.15.85 smithi085.front.sepia.ceph.com smithi085 2021-02-23T02:59:40.780 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120 2021-02-23T02:59:41.225 INFO:teuthology.provision.fog.smithi085:Deploy complete! 2021-02-23T02:59:41.255 INFO:teuthology.provision.fog.smithi120:Deploy complete! 2021-02-23T02:59:44.106 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-23T02:59:44.720 INFO:teuthology.provision.fog.smithi121:Node is ready 2021-02-23T02:59:44.857 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121 2021-02-23T02:59:44.939 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121 2021-02-23T02:59:45.444 INFO:teuthology.provision.fog.smithi121:Deploy complete! 2021-02-23T02:59:49.182 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-23T02:59:49.769 INFO:teuthology.provision.fog.smithi080:Node is ready 2021-02-23T02:59:49.854 INFO:teuthology.orchestra.run.smithi080.stdout:smithi080.front.sepia.ceph.com 2021-02-23T02:59:49.926 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80 smithi080.front.sepia.ceph.com smithi080 2021-02-23T02:59:50.435 INFO:teuthology.provision.fog.smithi080:Deploy complete! 2021-02-23T03:00:00.029 INFO:teuthology.provision.fog.smithi172:Node is ready 2021-02-23T03:00:00.153 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172 2021-02-23T03:00:00.238 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2021-02-23T03:00:00.720 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2021-02-23T03:00:01.030 INFO:teuthology.lock.ops:Checking smithi080.front.sepia.ceph.com 2021-02-23T03:00:01.035 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-23T03:00:01.051 INFO:teuthology.lock.ops:Checking smithi085.front.sepia.ceph.com 2021-02-23T03:00:01.051 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-23T03:00:01.062 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com 2021-02-23T03:00:01.063 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-23T03:00:01.075 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com 2021-02-23T03:00:01.076 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-23T03:00:01.098 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2021-02-23T03:00:01.099 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-23T03:00:01.114 INFO:teuthology.lock.ops:Updating [smithi080.front.sepia.ceph.com]: set os type and version on server 2021-02-23T03:00:01.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-23T03:00:01.701 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2021-02-23T03:00:01.782 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="CentOS Linux" 2021-02-23T03:00:01.783 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="7 (Core)" 2021-02-23T03:00:01.783 INFO:teuthology.orchestra.run.smithi080.stdout:ID="centos" 2021-02-23T03:00:01.784 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE="rhel fedora" 2021-02-23T03:00:01.784 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="7" 2021-02-23T03:00:01.785 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-02-23T03:00:01.785 INFO:teuthology.orchestra.run.smithi080.stdout:ANSI_COLOR="0;31" 2021-02-23T03:00:01.786 INFO:teuthology.orchestra.run.smithi080.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-02-23T03:00:01.786 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.centos.org/" 2021-02-23T03:00:01.786 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-23T03:00:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: 2021-02-23T03:00:01.787 INFO:teuthology.orchestra.run.smithi080.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-02-23T03:00:01.788 INFO:teuthology.orchestra.run.smithi080.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-02-23T03:00:01.788 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-23T03:00:01.788 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-02-23T03:00:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: 2021-02-23T03:00:01.793 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2021-02-23T03:00:01.815 INFO:teuthology.lock.ops:Updating [smithi085.front.sepia.ceph.com]: set os type and version on server 2021-02-23T03:00:01.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-23T03:00:02.408 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64 2021-02-23T03:00:02.480 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="CentOS Linux" 2021-02-23T03:00:02.480 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="7 (Core)" 2021-02-23T03:00:02.481 INFO:teuthology.orchestra.run.smithi085.stdout:ID="centos" 2021-02-23T03:00:02.481 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE="rhel fedora" 2021-02-23T03:00:02.481 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="7" 2021-02-23T03:00:02.481 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-02-23T03:00:02.481 INFO:teuthology.orchestra.run.smithi085.stdout:ANSI_COLOR="0;31" 2021-02-23T03:00:02.482 INFO:teuthology.orchestra.run.smithi085.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-02-23T03:00:02.482 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://www.centos.org/" 2021-02-23T03:00:02.482 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-23T03:00:02.482 INFO:teuthology.orchestra.run.smithi085.stdout: 2021-02-23T03:00:02.482 INFO:teuthology.orchestra.run.smithi085.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-02-23T03:00:02.483 INFO:teuthology.orchestra.run.smithi085.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-02-23T03:00:02.483 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-23T03:00:02.483 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-02-23T03:00:02.483 INFO:teuthology.orchestra.run.smithi085.stdout: 2021-02-23T03:00:02.484 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2021-02-23T03:00:02.497 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server 2021-02-23T03:00:02.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-23T03:00:03.084 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2021-02-23T03:00:03.146 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Linux" 2021-02-23T03:00:03.147 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="7 (Core)" 2021-02-23T03:00:03.148 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2021-02-23T03:00:03.148 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2021-02-23T03:00:03.148 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="7" 2021-02-23T03:00:03.149 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-02-23T03:00:03.149 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2021-02-23T03:00:03.150 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-02-23T03:00:03.150 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.centos.org/" 2021-02-23T03:00:03.151 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-23T03:00:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: 2021-02-23T03:00:03.151 INFO:teuthology.orchestra.run.smithi120.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-02-23T03:00:03.152 INFO:teuthology.orchestra.run.smithi120.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-02-23T03:00:03.152 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-23T03:00:03.153 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-02-23T03:00:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: 2021-02-23T03:00:03.154 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2021-02-23T03:00:03.172 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server 2021-02-23T03:00:03.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-23T03:00:03.753 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2021-02-23T03:00:03.825 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Linux" 2021-02-23T03:00:03.825 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="7 (Core)" 2021-02-23T03:00:03.826 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2021-02-23T03:00:03.826 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2021-02-23T03:00:03.826 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="7" 2021-02-23T03:00:03.826 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-02-23T03:00:03.827 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2021-02-23T03:00:03.827 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-02-23T03:00:03.827 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.centos.org/" 2021-02-23T03:00:03.827 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-23T03:00:03.827 INFO:teuthology.orchestra.run.smithi121.stdout: 2021-02-23T03:00:03.828 INFO:teuthology.orchestra.run.smithi121.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-02-23T03:00:03.828 INFO:teuthology.orchestra.run.smithi121.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-02-23T03:00:03.828 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-23T03:00:03.828 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-02-23T03:00:03.829 INFO:teuthology.orchestra.run.smithi121.stdout: 2021-02-23T03:00:03.829 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2021-02-23T03:00:03.845 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2021-02-23T03:00:03.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-23T03:00:04.400 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2021-02-23T03:00:04.475 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Linux" 2021-02-23T03:00:04.476 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="7 (Core)" 2021-02-23T03:00:04.476 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2021-02-23T03:00:04.476 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2021-02-23T03:00:04.476 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="7" 2021-02-23T03:00:04.477 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-02-23T03:00:04.477 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2021-02-23T03:00:04.477 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-02-23T03:00:04.477 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.centos.org/" 2021-02-23T03:00:04.478 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-23T03:00:04.478 INFO:teuthology.orchestra.run.smithi172.stdout: 2021-02-23T03:00:04.480 INFO:teuthology.orchestra.run.smithi172.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-02-23T03:00:04.480 INFO:teuthology.orchestra.run.smithi172.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-02-23T03:00:04.480 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-23T03:00:04.481 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-02-23T03:00:04.481 INFO:teuthology.orchestra.run.smithi172.stdout: 2021-02-23T03:00:04.482 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2021-02-23T03:00:04.625 INFO:teuthology.dispatcher.supervisor:Running job 5905597 2021-02-23T03:00:04.626 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-02-23_02:30:05-upgrade:mimic-x-nautilus-distro-basic-smithi/5905597 --name teuthology-2021-02-23_02:30:05-upgrade:mimic-x-nautilus-distro-basic-smithi --description upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} -- /home/teuthworker/archive/teuthology-2021-02-23_02:30:05-upgrade:mimic-x-nautilus-distro-basic-smithi/5905597/orig.config.yaml 2021-02-23T03:00:04.633 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-02-23_02:30:05-upgrade:mimic-x-nautilus-distro-basic-smithi/5905597 2021-02-23T03:00:04.633 INFO:teuthology.dispatcher.supervisor:Job PID: 24231 2021-02-23T03:00:04.634 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-23T15:00:21.993 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-23T15:00:53.967 INFO:teuthology.kill:Killing Pids: {24231} 2021-02-23T15:00:56.152 INFO:teuthology.kill:Nuking machines: ['smithi080', 'smithi085', 'smithi120', 'smithi121', 'smithi172'] 2021-02-23T15:00:57.600 INFO:teuthology.kill:2021-02-23 15:00:57,600.600 INFO:teuthology.nuke:targets: 2021-02-23T15:00:57.601 INFO:teuthology.kill: smithi080.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMhm5bcw158cKig+YoQxt0Cru/m4VEZ7i4F6GLSd4XjmRyV32C0Ibkm9n5xW03IgMeoDdMwTiIjnmrsuLvYfRgSPzHuDc4R1Lt6xayKEAuLUgSMp63nVykZ5F/wEX4IyzuGLXqt20u05+ruhs3nY+z7B6UOOcQnNt9AOAx5HpIMihK61mHgf9CTqg/SiSkaR6/wiaCO1kThlDN6w7SNv4A7uiqyE9fSl6bGbOZrICLtSheBktgPbbVCDJgW446QIGc30YHbpr89w/4N630+QYr3SlrN1pEGzRrHdEsM2L9rXkArmRGjkAFkd1vUfMFZLEsMubkHaFhTmPQDrGc+sWx 2021-02-23T15:00:57.601 INFO:teuthology.kill: smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoRnd9NYzG9vVUcUSTp71UAGEIIBBt1P/M4+kN/3opZUzaGfl0nrECoiAHIrN9TX/KKRmEjNB225tNCV1iVJ8T0qMt1gMBbQkk9TMKPvjZXXqwHnUxTNlJcFYK40uXy8bPVFdXLuVjrCcsDDgvzZkwcJQQBzNU4NRrSdM7sJl2jwVkw7mg8SVzgloNdqi3N8VAS6iKDWE+nOzPlFz+4gOyvIzRGrD7BlRcQlpBDNMrlwvYZTv7Bhd0QyiHlr1iRTJfqoIMEi37WE+V+patxHnrsdZOVnxl1MYWgC/p5F8gDnffkihMbuu3bSUWqlmg22MODZZSMvpd80maxUss6r/p 2021-02-23T15:00:57.602 INFO:teuthology.kill: smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC65YJ/JSBCKyKfF8wtGAEVu/XfzH4r2BOPnUQ15zWNpZw9vmY90vkbFI/q/Lmpdnta7e7TVkKDZuXqir2kxFLLFIbSVuDeaK89BrSuYDxOhp8YUg+YeEQILIBH1YG/4GXcOiBR0+RcK6sygdbbqbKSJRKzg8vg2EwKMNJKUiip+y0bpuBrdTERGQ9MXL4zJlmTWfn88i8SOnyA3unrmXbDY6bpSHkzZ3M8SFfzkYvCIqRka+vpWWqmUKWzORRrqk8/gQVMRNnsvHAdBriyo2sMnSdBLwJJzdZEyEx6jpcRgrEi6y1HsPEMXSFFW78DvH0/acS/gt4jeotp7n3iFQf3 2021-02-23T15:00:57.602 INFO:teuthology.kill: smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDv5nd6iCBB48c7lOUw0j31y/59wigWPZdh3o4J6RmQmSsmxmFXuT3bIc+dtNxoffVedzAyGfZUyQAaDjpdxp78Fwl2ri0pgqBCvC4rOU81OjjbKbwd/WevzMkltn44m9Xqmc9Z0tWOLbBSqfEq9bwY23u3DHFP36cAO9xMQk6A+sRhjkhNGPUg/us/HO7Fz1nqUBPI49Wyg0W7PUh0pqw4GvvGAxX5IQ84Uny2FfPOzm2iWwSkiU6UMAPu0ytiXlneqclS40KK2S++NU4zLWF38qcNxeLXpKUnDRTvRVLSS4JwhZ1cDzodbXCKrdYqyqIRTpprz+TgkKfPZHlzavr3 2021-02-23T15:00:57.603 INFO:teuthology.kill: smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpRNJ766Jpe5JSBukWqzRSFsW94c1AXv6Kx0jDBdwkUTT5nu2leUjlnHYpUesseHABM8/bouDJ1LSwqz9Lnn3YRcb0t+tLAL0rKOTUrktecOYfzoi/gguc1M3H/7Km2jvJKWqeWDB1D7sn79imM6n5qLdW6lv2/gvqw7eKI9hZxZl31rYCGL+CIofMgCZWLXVZ5ksaNrMV2+kjuAcLe6FWXWkqTtejm4BIH3/wTZicaCgXMXXCbhW7sSCGcyowW8aiewI/jK0OYrjLvc/y+fQ8welC4sh0Vw8Vqb5fXSUBtYdvbd1Aj7P+biiv0I97kT6vJzzRbczpwcVv33uAH4KH 2021-02-23T15:00:57.604 INFO:teuthology.kill:2021-02-23 15:00:57,600.600 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:00:57.612 INFO:teuthology.kill:2021-02-23 15:00:57,612.612 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:00:57.614 INFO:teuthology.kill:2021-02-23 15:00:57,614.614 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:00:57.616 INFO:teuthology.kill:2021-02-23 15:00:57,616.616 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:00:57.618 INFO:teuthology.kill:2021-02-23 15:00:57,618.618 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:00:57.698 INFO:teuthology.kill:2021-02-23 15:00:57,697.697 INFO:teuthology.orchestra.console:Power off smithi172 2021-02-23T15:00:57.719 INFO:teuthology.kill:2021-02-23 15:00:57,719.719 INFO:teuthology.orchestra.console:Power off smithi080 2021-02-23T15:00:57.734 INFO:teuthology.kill:2021-02-23 15:00:57,733.733 INFO:teuthology.orchestra.console:Power off smithi121 2021-02-23T15:00:57.747 INFO:teuthology.kill:2021-02-23 15:00:57,746.746 INFO:teuthology.orchestra.console:Power off smithi120 2021-02-23T15:00:57.779 INFO:teuthology.kill:2021-02-23 15:00:57,779.779 INFO:teuthology.orchestra.console:Power off smithi085 2021-02-23T15:01:06.131 INFO:teuthology.kill:2021-02-23 15:01:06,130.130 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2021-02-23T15:01:06.139 INFO:teuthology.kill:2021-02-23 15:01:06,139.139 INFO:teuthology.orchestra.console:Power off for smithi080 completed 2021-02-23T15:01:06.147 INFO:teuthology.kill:2021-02-23 15:01:06,147.147 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2021-02-23T15:01:06.153 INFO:teuthology.kill:2021-02-23 15:01:06,153.153 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2021-02-23T15:01:06.160 INFO:teuthology.kill:2021-02-23 15:01:06,160.160 INFO:teuthology.orchestra.console:Power off for smithi085 completed 2021-02-23T15:01:06.590 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2021-02-23T15:01:06.595 INFO:teuthology.task.internal:roles: ubuntu@smithi085.front.sepia.ceph.com - ['mon.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-02-23T15:01:06.595 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.c'] 2021-02-23T15:01:06.595 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11'] 2021-02-23T15:01:06.596 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['client.0'] 2021-02-23T15:01:06.600 INFO:teuthology.misc:Compressing logs... 2021-02-23T15:01:06.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-23T15:02:06.742 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-23T15:02:33.055 INFO:teuthology.kill:No teuthology processes running 2021-02-23T15:02:36.251 INFO:teuthology.kill:Nuking machines: ['smithi080', 'smithi085', 'smithi120', 'smithi121', 'smithi172'] 2021-02-23T15:02:38.343 INFO:teuthology.kill:2021-02-23 15:02:38,343.343 INFO:teuthology.nuke:targets: 2021-02-23T15:02:38.346 INFO:teuthology.kill: smithi080.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMhm5bcw158cKig+YoQxt0Cru/m4VEZ7i4F6GLSd4XjmRyV32C0Ibkm9n5xW03IgMeoDdMwTiIjnmrsuLvYfRgSPzHuDc4R1Lt6xayKEAuLUgSMp63nVykZ5F/wEX4IyzuGLXqt20u05+ruhs3nY+z7B6UOOcQnNt9AOAx5HpIMihK61mHgf9CTqg/SiSkaR6/wiaCO1kThlDN6w7SNv4A7uiqyE9fSl6bGbOZrICLtSheBktgPbbVCDJgW446QIGc30YHbpr89w/4N630+QYr3SlrN1pEGzRrHdEsM2L9rXkArmRGjkAFkd1vUfMFZLEsMubkHaFhTmPQDrGc+sWx 2021-02-23T15:02:38.346 INFO:teuthology.kill: smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoRnd9NYzG9vVUcUSTp71UAGEIIBBt1P/M4+kN/3opZUzaGfl0nrECoiAHIrN9TX/KKRmEjNB225tNCV1iVJ8T0qMt1gMBbQkk9TMKPvjZXXqwHnUxTNlJcFYK40uXy8bPVFdXLuVjrCcsDDgvzZkwcJQQBzNU4NRrSdM7sJl2jwVkw7mg8SVzgloNdqi3N8VAS6iKDWE+nOzPlFz+4gOyvIzRGrD7BlRcQlpBDNMrlwvYZTv7Bhd0QyiHlr1iRTJfqoIMEi37WE+V+patxHnrsdZOVnxl1MYWgC/p5F8gDnffkihMbuu3bSUWqlmg22MODZZSMvpd80maxUss6r/p 2021-02-23T15:02:38.346 INFO:teuthology.kill: smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC65YJ/JSBCKyKfF8wtGAEVu/XfzH4r2BOPnUQ15zWNpZw9vmY90vkbFI/q/Lmpdnta7e7TVkKDZuXqir2kxFLLFIbSVuDeaK89BrSuYDxOhp8YUg+YeEQILIBH1YG/4GXcOiBR0+RcK6sygdbbqbKSJRKzg8vg2EwKMNJKUiip+y0bpuBrdTERGQ9MXL4zJlmTWfn88i8SOnyA3unrmXbDY6bpSHkzZ3M8SFfzkYvCIqRka+vpWWqmUKWzORRrqk8/gQVMRNnsvHAdBriyo2sMnSdBLwJJzdZEyEx6jpcRgrEi6y1HsPEMXSFFW78DvH0/acS/gt4jeotp7n3iFQf3 2021-02-23T15:02:38.347 INFO:teuthology.kill: smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDv5nd6iCBB48c7lOUw0j31y/59wigWPZdh3o4J6RmQmSsmxmFXuT3bIc+dtNxoffVedzAyGfZUyQAaDjpdxp78Fwl2ri0pgqBCvC4rOU81OjjbKbwd/WevzMkltn44m9Xqmc9Z0tWOLbBSqfEq9bwY23u3DHFP36cAO9xMQk6A+sRhjkhNGPUg/us/HO7Fz1nqUBPI49Wyg0W7PUh0pqw4GvvGAxX5IQ84Uny2FfPOzm2iWwSkiU6UMAPu0ytiXlneqclS40KK2S++NU4zLWF38qcNxeLXpKUnDRTvRVLSS4JwhZ1cDzodbXCKrdYqyqIRTpprz+TgkKfPZHlzavr3 2021-02-23T15:02:38.347 INFO:teuthology.kill: smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpRNJ766Jpe5JSBukWqzRSFsW94c1AXv6Kx0jDBdwkUTT5nu2leUjlnHYpUesseHABM8/bouDJ1LSwqz9Lnn3YRcb0t+tLAL0rKOTUrktecOYfzoi/gguc1M3H/7Km2jvJKWqeWDB1D7sn79imM6n5qLdW6lv2/gvqw7eKI9hZxZl31rYCGL+CIofMgCZWLXVZ5ksaNrMV2+kjuAcLe6FWXWkqTtejm4BIH3/wTZicaCgXMXXCbhW7sSCGcyowW8aiewI/jK0OYrjLvc/y+fQ8welC4sh0Vw8Vqb5fXSUBtYdvbd1Aj7P+biiv0I97kT6vJzzRbczpwcVv33uAH4KH 2021-02-23T15:02:38.380 INFO:teuthology.kill:2021-02-23 15:02:38,380.380 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:02:38.384 INFO:teuthology.kill:2021-02-23 15:02:38,384.384 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:02:38.387 INFO:teuthology.kill:2021-02-23 15:02:38,387.387 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:02:38.391 INFO:teuthology.kill:2021-02-23 15:02:38,391.391 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:02:38.394 INFO:teuthology.kill:2021-02-23 15:02:38,394.394 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-23T15:02:38.456 INFO:teuthology.kill:2021-02-23 15:02:38,455.455 INFO:teuthology.orchestra.console:Power off smithi121 2021-02-23T15:02:38.475 INFO:teuthology.kill:2021-02-23 15:02:38,475.475 INFO:teuthology.orchestra.console:Power off smithi120 2021-02-23T15:02:38.490 INFO:teuthology.kill:2021-02-23 15:02:38,490.490 INFO:teuthology.orchestra.console:Power off smithi080 2021-02-23T15:02:38.505 INFO:teuthology.kill:2021-02-23 15:02:38,504.504 INFO:teuthology.orchestra.console:Power off smithi172 2021-02-23T15:02:38.524 INFO:teuthology.kill:2021-02-23 15:02:38,524.524 INFO:teuthology.orchestra.console:Power off smithi085 2021-02-23T15:02:38.671 INFO:teuthology.kill:2021-02-23 15:02:38,671.671 INFO:teuthology.orchestra.console:Power off for smithi080 completed 2021-02-23T15:02:38.831 INFO:teuthology.kill:2021-02-23 15:02:38,671.671 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2021-02-23T15:02:38.831 INFO:teuthology.kill:2021-02-23 15:02:38,672.672 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2021-02-23T15:02:38.832 INFO:teuthology.kill:2021-02-23 15:02:38,677.677 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2021-02-23T15:02:38.832 INFO:teuthology.kill:2021-02-23 15:02:38,683.683 INFO:teuthology.orchestra.console:Power off for smithi085 completed 2021-02-23T15:02:38.832 INFO:teuthology.kill:2021-02-23 15:02:38,825.825 INFO:teuthology.lock.ops:unlocked smithi172.front.sepia.ceph.com 2021-02-23T15:02:38.832 INFO:teuthology.kill:2021-02-23 15:02:38,828.828 INFO:teuthology.lock.ops:unlocked smithi121.front.sepia.ceph.com 2021-02-23T15:02:38.833 INFO:teuthology.kill:2021-02-23 15:02:38,829.829 INFO:teuthology.lock.ops:unlocked smithi085.front.sepia.ceph.com 2021-02-23T15:02:38.833 INFO:teuthology.kill:2021-02-23 15:02:38,830.830 INFO:teuthology.lock.ops:unlocked smithi080.front.sepia.ceph.com 2021-02-23T15:02:38.833 INFO:teuthology.kill:2021-02-23 15:02:38,832.832 INFO:teuthology.lock.ops:unlocked smithi120.front.sepia.ceph.com 2021-02-23T15:04:39.020 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-23T15:05:19.336 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-23T15:05:19.367 INFO:teuthology.nuke:Checking targets against current locks 2021-02-23T15:05:19.704 INFO:teuthology.nuke:Not nuking smithi080.front.sepia.ceph.com because description doesn't match 2021-02-23T15:05:19.802 INFO:teuthology.nuke:Not nuking smithi121.front.sepia.ceph.com because description doesn't match 2021-02-23T15:05:19.807 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 59, in main verbose File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 152, in run_job unlock_targets(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 196, in unlock_targets nuke(fake_ctx, True) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/nuke/__init__.py", line 250, in nuke if ctx.name not in lock['description']: TypeError: argument of type 'NoneType' is not iterable