2021-02-09T05:00:03.202 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-09T05:00:03.400 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi115.front.sepia.ceph.com' reimaging 2021-02-09T05:00:03.401 INFO:teuthology.lock.ops:Updating [ubuntu@smithi115.front.sepia.ceph.com]: reset os type and version on server 2021-02-09T05:00:03.401 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2021-02-09T05:00:03.418 INFO:teuthology.lock.ops:Node 'ubuntu@smithi115.front.sepia.ceph.com' reimaging is complete 2021-02-09T05:00:03.418 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging 2021-02-09T05:00:03.418 INFO:teuthology.lock.ops:Updating [ubuntu@smithi136.front.sepia.ceph.com]: reset os type and version on server 2021-02-09T05:00:03.419 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-02-09T05:00:03.442 INFO:teuthology.lock.ops:Node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging is complete 2021-02-09T05:00:03.443 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging 2021-02-09T05:00:03.443 INFO:teuthology.lock.ops:Updating [ubuntu@smithi155.front.sepia.ceph.com]: reset os type and version on server 2021-02-09T05:00:03.444 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2021-02-09T05:00:03.474 INFO:teuthology.lock.ops:Node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging is complete 2021-02-09T05:00:03.474 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi184.front.sepia.ceph.com' reimaging 2021-02-09T05:00:03.475 INFO:teuthology.lock.ops:Updating [ubuntu@smithi184.front.sepia.ceph.com]: reset os type and version on server 2021-02-09T05:00:03.476 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2021-02-09T05:00:03.497 INFO:teuthology.lock.ops:Node 'ubuntu@smithi184.front.sepia.ceph.com' reimaging is complete 2021-02-09T05:00:03.497 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging 2021-02-09T05:00:03.498 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server 2021-02-09T05:00:03.498 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2021-02-09T05:00:03.533 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete 2021-02-09T05:00:03.662 INFO:teuthology.provision.fog.smithi115:Scheduling deploy of ubuntu 18.04 2021-02-09T05:00:03.701 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of ubuntu 18.04 2021-02-09T05:00:03.742 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of ubuntu 18.04 2021-02-09T05:00:03.745 INFO:teuthology.provision.fog.smithi184:Scheduling deploy of ubuntu 18.04 2021-02-09T05:00:03.783 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of ubuntu 18.04 2021-02-09T05:00:04.070 INFO:teuthology.orchestra.console:Power off smithi115 2021-02-09T05:00:04.116 INFO:teuthology.orchestra.console:Power off smithi136 2021-02-09T05:00:04.219 INFO:teuthology.orchestra.console:Power off smithi155 2021-02-09T05:00:04.259 INFO:teuthology.orchestra.console:Power off smithi184 2021-02-09T05:00:04.271 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2021-02-09T05:00:04.271 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-02-09T05:00:04.319 INFO:teuthology.orchestra.console:Power off smithi185 2021-02-09T05:00:04.372 INFO:teuthology.orchestra.console:Power on smithi115 2021-02-09T05:00:04.382 INFO:teuthology.orchestra.console:Power on smithi136 2021-02-09T05:00:04.395 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2021-02-09T05:00:04.434 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2021-02-09T05:00:04.470 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2021-02-09T05:00:04.497 INFO:teuthology.orchestra.console:Power on smithi155 2021-02-09T05:00:04.537 INFO:teuthology.orchestra.console:Power on smithi184 2021-02-09T05:00:04.575 INFO:teuthology.orchestra.console:Power on smithi185 2021-02-09T05:00:08.803 INFO:teuthology.orchestra.console:Power on for smithi184 completed 2021-02-09T05:00:08.864 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2021-02-09T05:00:08.904 INFO:teuthology.provision.fog.smithi184:Waiting for deploy to finish 2021-02-09T05:00:08.965 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2021-02-09T05:00:12.752 INFO:teuthology.orchestra.console:Power on for smithi115 completed 2021-02-09T05:00:12.762 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2021-02-09T05:00:12.854 INFO:teuthology.provision.fog.smithi115:Waiting for deploy to finish 2021-02-09T05:00:12.864 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2021-02-09T05:00:12.876 INFO:teuthology.orchestra.console:Power on for smithi155 completed 2021-02-09T05:00:12.977 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2021-02-09T05:05:10.741 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-09T05:05:11.269 INFO:teuthology.provision.fog.smithi184:Node is ready 2021-02-09T05:05:11.315 INFO:teuthology.orchestra.run.smithi184.stdout:smithi184.front.sepia.ceph.com 2021-02-09T05:05:11.363 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.15.184 smithi184.front.sepia.ceph.com smithi184 2021-02-09T05:05:11.685 INFO:teuthology.provision.fog.smithi184:Deploy complete! 2021-02-09T05:05:14.505 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-09T05:05:15.091 INFO:teuthology.provision.fog.smithi136:Node is ready 2021-02-09T05:05:15.099 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2021-02-09T05:05:15.155 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2021-02-09T05:05:15.458 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2021-02-09T05:05:23.545 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-09T05:05:23.634 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-09T05:05:24.021 INFO:teuthology.provision.fog.smithi115:Node is ready 2021-02-09T05:05:24.115 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115.front.sepia.ceph.com 2021-02-09T05:05:24.145 INFO:teuthology.provision.fog.smithi155:Node is ready 2021-02-09T05:05:24.165 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115 smithi115.front.sepia.ceph.com smithi115 2021-02-09T05:05:24.193 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2021-02-09T05:05:24.242 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2021-02-09T05:05:24.458 INFO:teuthology.provision.fog.smithi115:Deploy complete! 2021-02-09T05:05:24.552 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2021-02-09T05:05:34.842 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-09T05:05:35.384 INFO:teuthology.provision.fog.smithi185:Node is ready 2021-02-09T05:05:35.433 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2021-02-09T05:05:35.482 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2021-02-09T05:05:35.772 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2021-02-09T05:05:36.030 INFO:teuthology.lock.ops:Checking smithi115.front.sepia.ceph.com 2021-02-09T05:05:36.030 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-09T05:05:36.047 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2021-02-09T05:05:36.048 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-09T05:05:36.061 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2021-02-09T05:05:36.062 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-09T05:05:36.075 INFO:teuthology.lock.ops:Checking smithi184.front.sepia.ceph.com 2021-02-09T05:05:36.075 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-09T05:05:36.089 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2021-02-09T05:05:36.089 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-09T05:05:36.104 INFO:teuthology.lock.ops:Updating [smithi115.front.sepia.ceph.com]: set os type and version on server 2021-02-09T05:05:36.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T05:05:36.472 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2021-02-09T05:05:36.522 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Ubuntu" 2021-02-09T05:05:36.522 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-09T05:05:36.522 INFO:teuthology.orchestra.run.smithi115.stdout:ID=ubuntu 2021-02-09T05:05:36.522 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE=debian 2021-02-09T05:05:36.522 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-09T05:05:36.522 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="18.04" 2021-02-09T05:05:36.522 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-09T05:05:36.522 INFO:teuthology.orchestra.run.smithi115.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-09T05:05:36.523 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-09T05:05:36.523 INFO:teuthology.orchestra.run.smithi115.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-09T05:05:36.523 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_CODENAME=bionic 2021-02-09T05:05:36.523 INFO:teuthology.orchestra.run.smithi115.stdout:UBUNTU_CODENAME=bionic 2021-02-09T05:05:36.524 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2021-02-09T05:05:36.538 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2021-02-09T05:05:36.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T05:05:36.910 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2021-02-09T05:05:36.958 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2021-02-09T05:05:36.958 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-09T05:05:36.958 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2021-02-09T05:05:36.958 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2021-02-09T05:05:36.958 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-09T05:05:36.958 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="18.04" 2021-02-09T05:05:36.958 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-09T05:05:36.959 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-09T05:05:36.959 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-09T05:05:36.959 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-09T05:05:36.959 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=bionic 2021-02-09T05:05:36.959 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=bionic 2021-02-09T05:05:36.959 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-02-09T05:05:36.972 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2021-02-09T05:05:36.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T05:05:37.337 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2021-02-09T05:05:37.385 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Ubuntu" 2021-02-09T05:05:37.385 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-09T05:05:37.385 INFO:teuthology.orchestra.run.smithi155.stdout:ID=ubuntu 2021-02-09T05:05:37.385 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE=debian 2021-02-09T05:05:37.385 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-09T05:05:37.385 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="18.04" 2021-02-09T05:05:37.386 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-09T05:05:37.386 INFO:teuthology.orchestra.run.smithi155.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-09T05:05:37.386 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-09T05:05:37.386 INFO:teuthology.orchestra.run.smithi155.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-09T05:05:37.386 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_CODENAME=bionic 2021-02-09T05:05:37.386 INFO:teuthology.orchestra.run.smithi155.stdout:UBUNTU_CODENAME=bionic 2021-02-09T05:05:37.386 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2021-02-09T05:05:37.402 INFO:teuthology.lock.ops:Updating [smithi184.front.sepia.ceph.com]: set os type and version on server 2021-02-09T05:05:37.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T05:05:37.763 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2021-02-09T05:05:37.810 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="Ubuntu" 2021-02-09T05:05:37.810 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-09T05:05:37.810 INFO:teuthology.orchestra.run.smithi184.stdout:ID=ubuntu 2021-02-09T05:05:37.810 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE=debian 2021-02-09T05:05:37.810 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-09T05:05:37.811 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="18.04" 2021-02-09T05:05:37.811 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-09T05:05:37.811 INFO:teuthology.orchestra.run.smithi184.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-09T05:05:37.811 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-09T05:05:37.811 INFO:teuthology.orchestra.run.smithi184.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-09T05:05:37.811 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_CODENAME=bionic 2021-02-09T05:05:37.811 INFO:teuthology.orchestra.run.smithi184.stdout:UBUNTU_CODENAME=bionic 2021-02-09T05:05:37.811 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2021-02-09T05:05:37.826 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2021-02-09T05:05:37.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T05:05:38.196 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Ubuntu" 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:ID=ubuntu 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE=debian 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="18.04" 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-09T05:05:38.259 INFO:teuthology.orchestra.run.smithi185.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-09T05:05:38.260 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_CODENAME=bionic 2021-02-09T05:05:38.260 INFO:teuthology.orchestra.run.smithi185.stdout:UBUNTU_CODENAME=bionic 2021-02-09T05:05:38.260 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2021-02-09T05:05:38.366 INFO:teuthology.dispatcher.supervisor:Running job 5869871 2021-02-09T05:05:38.366 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-02-09_02:30:03-upgrade:mimic-x-nautilus-distro-basic-smithi/5869871 --name teuthology-2021-02-09_02:30:03-upgrade:mimic-x-nautilus-distro-basic-smithi --description upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} -- /home/teuthworker/archive/teuthology-2021-02-09_02:30:03-upgrade:mimic-x-nautilus-distro-basic-smithi/5869871/orig.config.yaml 2021-02-09T05:05:38.373 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-02-09_02:30:03-upgrade:mimic-x-nautilus-distro-basic-smithi/5869871 2021-02-09T05:05:38.373 INFO:teuthology.dispatcher.supervisor:Job PID: 3073 2021-02-09T05:05:38.374 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-09T17:05:54.148 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-09T17:05:56.932 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2021-02-09T17:05:56.932 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-02-09T17:05:56.932 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.c'] 2021-02-09T17:05:56.932 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11'] 2021-02-09T17:05:56.932 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['client.0'] 2021-02-09T17:05:56.933 INFO:teuthology.misc:Compressing logs... 2021-02-09T17:05:56.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T17:05:57.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T17:05:58.042 INFO:teuthology.orchestra.run.smithi115.stderr: 2021-02-09T17:05:58.042 INFO:teuthology.orchestra.run.smithi115.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/kern.log.gz: No space left on device 2021-02-09T17:05:58.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T17:05:58.952 INFO:teuthology.orchestra.run.smithi136.stderr: 2021-02-09T17:05:58.952 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/kern.log.gz: No space left on device 2021-02-09T17:05:59.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T17:05:59.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T17:06:00.287 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/dispatcher/supervisor.py", line 220, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/dispatcher/supervisor.py", line 274, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/misc.py", line 1372, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/orchestra/run.py", line 183, in _raise_for_status node=self.hostname, label=self.label teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 123: 'sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --' 2021-02-09T17:06:00.438 INFO:teuthology.kill:Killing Pids: {3073} 2021-02-09T17:06:02.086 INFO:teuthology.kill:Nuking machines: ['smithi115', 'smithi136', 'smithi155', 'smithi184', 'smithi185'] 2021-02-09T17:06:02.901 INFO:teuthology.kill:2021-02-09 17:06:02,901.901 INFO:teuthology.nuke:targets: 2021-02-09T17:06:02.902 INFO:teuthology.kill: smithi115.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8uSWQwc2BaU7Xb1tvcnQWrTda5pzxAcIN1tWTI/WothFlEEUNIP0EdEKVtf+ORW80r1Zt5BbdyCGAp/phVJOfbwgIeuW5m78x9dnV4WmLaUvCRr5Q/0ihI7hVxAB9d9DSfkFj92MlAROE8atpbDWTVhO459S4CYuS0gwskmANIFI6ovXgxeyw9O9FOjPM8us8n3t7nGHxsYdXPPaDPpGyky31DHBhJ5aSXzWtAdTcT5HtsWbMHijGB+HYUgVatEeFZ9Rl5gBBIXkyzx+p6urCvzIPsLmmi2skmXhaMrNg66viWTQDvQ3HnGe4UmnJSqzuRHBtz0FG1oufn5GPmAi5 2021-02-09T17:06:02.902 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDe2gbIHAHjw8a0W9zzI9RjPAfSDS3BSeSAUjUeT7yqB50Gnt9IdnshZPvdNdXTl0WQ5xiiPvWf3UHDxU6BvTSLbaI5oPqqODB9kVte8/5xxW2KRipChLj23MDQWHrp2lF6xK1g8filsscaZ4PTa0VXIhLmcU6hkupMy2o6OX+CI1PWiec4TjGC4om/VLZzMngushSc9ekjvqdlRxRqOTOSFZJslJ1xCW4/h0isQ0mmHj9EF7jOOhZjwoEwC8ZoOEySv2sAKBF7ihPWrJhxd5QM3gkIobKnvbAt4Oq6beqTo3L2kFKNx2ZYJX5f+KAZe1xhjgfGujFyde31IfAU854N 2021-02-09T17:06:02.902 INFO:teuthology.kill: smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDI+1rhbVGUEfNTiEomF76GsfjEqzNLD8Qgdk+TPTIV7IyFsIf/AEtxSxTW4ASpuxnp3oUGgseW/kvnZzrvShsa9w2JurOwTAYFgGXUPKpHFAzBl3Zx9InZUUImNxGRYHkYHECljBjeG8lENJXQ/vP0q3I/+1NP4E2BBfTUw/vR87htcYjlL1f84/z5wJ8xzXfnQg/CuxTkEBggJgqyjJqZTRjn+hYKcG7urs/I1ZUDXARouqanxWq4X2Ddxh7XxfuTrZ6uBhfZ1zntHQTwfS34frjXBFEK5nqBxdpGC3rUtJK2tlENNlcWUBiHgKLtU7PzohM0wCw1DTLmJq/SJ2L7 2021-02-09T17:06:02.902 INFO:teuthology.kill: smithi184.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5eHaLL7F6gNfkmPA5QRbkImVGct0NOrCL6hbl0n4jk9ujNI78ejIQsgT0wpWBUBww7rXO39dBAj592WcMdYvBLGD5bXc0TLawN0O/N5ImI2rSVVZEe9RdigvyxhXmqf7cH1svorDcS43i31T3uac1NvZvl+N8np/hv5smxqiUNwil5B5txcfbMikqcEsxwRmoxqU/0uf3jzKBMGPyZ+bljbgb8l4EnRI7FgChIsj/y2nHGmpLSgucex2unRO+L3ofCcqKkhGiaZa48YMJbcFBauyQVa7X7M/V+lk6/ET/2mkkGk2So/4eujqCCDz45SumhTDQzePCcrgID6M+62fp 2021-02-09T17:06:02.902 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCy8bdpvpBW0/+xQSk6pwnp2NU0vX/phplTjyVZvtkWSu9zi+XhnxsxxUe044bWcI1mjwpgXK8I/VaJncToYmAatULQy1klkwqLFTLrB1lhCvOHzVeNyXQbJkmYKaLkgFjyg53eca4I8PxmUm0DwF6/b5kXlZZnov4rhxvq8Vhcv/iwcepUFNREUIY02duLRSBhKJU8exc93/PO1UqK+xItFw1fwe9rEIBenSrZxXh5G2p5oSxLIreCuqiGKvt1vmGbyz4H8rxOB2dsXW/K8kSP3ME3pPTWnzKYwrwwuQDUZ5F6E20JHPpJFaY8S+KRg6HSjRvWvbh32Slnd2icICmD 2021-02-09T17:06:02.930 INFO:teuthology.kill:2021-02-09 17:06:02,929.929 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-09T17:06:02.932 INFO:teuthology.kill:2021-02-09 17:06:02,932.932 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-09T17:06:02.935 INFO:teuthology.kill:2021-02-09 17:06:02,935.935 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-09T17:06:02.939 INFO:teuthology.kill:2021-02-09 17:06:02,939.939 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-09T17:06:02.941 INFO:teuthology.kill:2021-02-09 17:06:02,941.941 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-09T17:06:03.015 INFO:teuthology.kill:2021-02-09 17:06:03,014.014 INFO:teuthology.orchestra.console:Power off smithi184 2021-02-09T17:06:03.032 INFO:teuthology.kill:2021-02-09 17:06:03,032.032 INFO:teuthology.orchestra.console:Power off smithi115 2021-02-09T17:06:03.047 INFO:teuthology.kill:2021-02-09 17:06:03,047.047 INFO:teuthology.orchestra.console:Power off smithi136 2021-02-09T17:06:03.061 INFO:teuthology.kill:2021-02-09 17:06:03,061.061 INFO:teuthology.orchestra.console:Power off smithi185 2021-02-09T17:06:03.073 INFO:teuthology.kill:2021-02-09 17:06:03,073.073 INFO:teuthology.orchestra.console:Power off smithi155 2021-02-09T17:06:11.474 INFO:teuthology.kill:2021-02-09 17:06:11,473.473 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2021-02-09T17:06:11.615 INFO:teuthology.kill:2021-02-09 17:06:11,614.614 INFO:teuthology.lock.ops:unlocked smithi155.front.sepia.ceph.com 2021-02-09T17:06:15.558 INFO:teuthology.kill:2021-02-09 17:06:15,558.558 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2021-02-09T17:06:15.564 INFO:teuthology.kill:2021-02-09 17:06:15,564.564 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-02-09T17:06:15.572 INFO:teuthology.kill:2021-02-09 17:06:15,571.571 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2021-02-09T17:06:15.579 INFO:teuthology.kill:2021-02-09 17:06:15,579.579 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2021-02-09T17:06:15.699 INFO:teuthology.kill:2021-02-09 17:06:15,699.699 INFO:teuthology.lock.ops:unlocked smithi115.front.sepia.ceph.com 2021-02-09T17:06:15.706 INFO:teuthology.kill:2021-02-09 17:06:15,706.706 INFO:teuthology.lock.ops:unlocked smithi136.front.sepia.ceph.com 2021-02-09T17:06:15.711 INFO:teuthology.kill:2021-02-09 17:06:15,711.711 INFO:teuthology.lock.ops:unlocked smithi185.front.sepia.ceph.com 2021-02-09T17:06:15.719 INFO:teuthology.kill:2021-02-09 17:06:15,718.718 INFO:teuthology.lock.ops:unlocked smithi184.front.sepia.ceph.com 2021-02-09T17:08:15.888 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-09T17:08:16.115 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-09T17:08:16.115 INFO:teuthology.nuke:Checking targets against current locks 2021-02-09T17:08:16.203 INFO:teuthology.nuke:Not nuking smithi184.front.sepia.ceph.com because description doesn't match 2021-02-09T17:08:16.203 INFO:teuthology.nuke:Not nuking smithi185.front.sepia.ceph.com because description doesn't match 2021-02-09T17:08:16.203 INFO:teuthology.nuke:Not nuking smithi155.front.sepia.ceph.com because description doesn't match 2021-02-09T17:08:16.203 INFO:teuthology.nuke:Not nuking smithi115.front.sepia.ceph.com because description doesn't match 2021-02-09T17:08:16.203 INFO:teuthology.nuke:Not nuking smithi136.front.sepia.ceph.com because description doesn't match