2021-02-10T02:01:53.085 INFO:root:teuthology version: 1.1.0-17c09965
2021-02-10T02:01:53.186 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging
2021-02-10T02:01:53.192 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server
2021-02-10T02:01:53.192 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2021-02-10T02:01:53.208 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete
2021-02-10T02:01:53.208 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging
2021-02-10T02:01:53.208 INFO:teuthology.lock.ops:Updating [ubuntu@smithi155.front.sepia.ceph.com]: reset os type and version on server
2021-02-10T02:01:53.209 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server
2021-02-10T02:01:53.222 INFO:teuthology.lock.ops:Node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging is complete
2021-02-10T02:01:53.422 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of rhel 8.3
2021-02-10T02:01:53.425 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of rhel 8.3
2021-02-10T02:01:53.803 INFO:teuthology.orchestra.console:Power off smithi155
2021-02-10T02:01:53.831 INFO:teuthology.orchestra.console:Power off smithi116
2021-02-10T02:02:02.199 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2021-02-10T02:02:02.300 INFO:teuthology.orchestra.console:Power on smithi116
2021-02-10T02:02:06.303 INFO:teuthology.orchestra.console:Power off for smithi155 completed
2021-02-10T02:02:06.404 INFO:teuthology.orchestra.console:Power on smithi155
2021-02-10T02:02:10.658 INFO:teuthology.orchestra.console:Power on for smithi155 completed
2021-02-10T02:02:10.669 INFO:teuthology.orchestra.console:Power on for smithi116 completed
2021-02-10T02:02:10.760 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish
2021-02-10T02:02:10.770 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish
2021-02-10T02:06:51.723 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-02-10T02:06:52.037 INFO:teuthology.provision.fog.smithi116:Node is ready
2021-02-10T02:06:52.051 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com
2021-02-10T02:06:52.105 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116
2021-02-10T02:06:52.426 INFO:teuthology.provision.fog.smithi116:Deploy complete!
2021-02-10T02:07:06.738 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-02-10T02:07:07.072 INFO:teuthology.provision.fog.smithi155:Node is ready
2021-02-10T02:07:07.091 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com
2021-02-10T02:07:07.146 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155
2021-02-10T02:07:07.492 INFO:teuthology.provision.fog.smithi155:Deploy complete!
2021-02-10T02:07:07.687 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com
2021-02-10T02:07:07.688 INFO:teuthology.lock.ops:New key found. Updating...
2021-02-10T02:07:07.727 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com
2021-02-10T02:07:07.727 INFO:teuthology.lock.ops:New key found. Updating...
2021-02-10T02:07:07.742 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server
2021-02-10T02:07:07.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-10T02:07:08.031 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64
2021-02-10T02:07:08.086 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Red Hat Enterprise Linux"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8.3 (Ootpa)"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:ID="rhel"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="fedora"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8.3"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-02-10T02:07:08.087 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.redhat.com/"
2021-02-10T02:07:08.088 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-02-10T02:07:08.088 INFO:teuthology.orchestra.run.smithi116.stdout:
2021-02-10T02:07:08.088 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-02-10T02:07:08.088 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-02-10T02:07:08.088 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-02-10T02:07:08.088 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-02-10T02:07:08.089 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2021-02-10T02:07:08.107 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server
2021-02-10T02:07:08.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-10T02:07:08.425 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64
2021-02-10T02:07:08.479 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux"
2021-02-10T02:07:08.479 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.3 (Ootpa)"
2021-02-10T02:07:08.479 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel"
2021-02-10T02:07:08.479 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora"
2021-02-10T02:07:08.479 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.3"
2021-02-10T02:07:08.479 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8"
2021-02-10T02:07:08.479 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31"
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/"
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-02-10T02:07:08.480 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-02-10T02:07:08.481 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server
2021-02-10T02:07:08.565 INFO:teuthology.dispatcher.supervisor:Running job 5871639
2021-02-10T02:07:08.565 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/virtualenv/bin/teuthology -v --owner scheduled_sseshasa@teuthology --archive /home/teuthworker/archive/sseshasa-2021-02-09_10:39:09-rados-wip-sseshasa-testing-2021-02-02-1136-distro-basic-smithi/5871639 --name sseshasa-2021-02-09_10:39:09-rados-wip-sseshasa-testing-2021-02-02-1136-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} -- /home/teuthworker/archive/sseshasa-2021-02-09_10:39:09-rados-wip-sseshasa-testing-2021-02-02-1136-distro-basic-smithi/5871639/orig.config.yaml
2021-02-10T02:07:08.571 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sseshasa-2021-02-09_10:39:09-rados-wip-sseshasa-testing-2021-02-02-1136-distro-basic-smithi/5871639
2021-02-10T02:07:08.572 INFO:teuthology.dispatcher.supervisor:Job PID: 10106
2021-02-10T02:07:08.572 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-02-10T14:07:24.666 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-02-10T14:07:24.904 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-02-10T14:07:24.904 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-02-10T14:07:24.904 INFO:teuthology.misc:Compressing logs...
2021-02-10T14:07:24.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-10T14:07:25.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-10T14:07:25.663 INFO:teuthology.misc:Compressing logs...
2021-02-10T14:07:26.606 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-02-10T14:07:26.896 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-02-10T14:10:01.096 INFO:teuthology.misc:Compressing logs...
2021-02-10T14:11:16.433 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/ceph-osd.5.log: file size changed while zipping
2021-02-10T14:12:11.451 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/ceph-mgr.x.log: file size changed while zipping
2021-02-10T14:12:38.889 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/ceph-mon.b.log: file size changed while zipping
2021-02-10T14:12:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/ceph-osd.0.log: file size changed while zipping
2021-02-10T14:14:05.003 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/ceph-osd.7.log: file size changed while zipping
2021-02-10T14:15:53.624 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/ceph-osd.2.log: file size changed while zipping
2021-02-10T14:16:53.415 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/ceph-mon.a.log: file size changed while zipping
2021-02-10T14:17:19.728 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/ceph-mon.c.log: file size changed while zipping
2021-02-10T14:17:20.237 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/ceph.log: file size changed while zipping
2021-02-10T14:18:36.777 INFO:teuthology.orchestra.run.smithi116.stderr:tar: ./ceph-osd.2.log: file changed as we read it
2021-02-10T14:18:41.428 INFO:teuthology.orchestra.run.smithi116.stderr:tar: ./ceph-osd.0.log: file changed as we read it
2021-02-10T14:19:36.574 INFO:teuthology.orchestra.run.smithi116.stderr:tar: .: file changed as we read it
2021-02-10T14:19:40.221 INFO:teuthology.orchestra.run.smithi155.stderr:tar: ./ceph-osd.5.log: file changed as we read it
2021-02-10T14:20:27.968 INFO:teuthology.orchestra.run.smithi155.stderr:tar: ./ceph-osd.7.log: file changed as we read it
2021-02-10T14:20:37.396 INFO:teuthology.kill:Killing Pids: {10106}
2021-02-10T14:20:39.341 INFO:teuthology.kill:Nuking machines: ['smithi116', 'smithi155']
2021-02-10T14:20:41.068 INFO:teuthology.kill:2021-02-10 14:20:41,068.068 INFO:teuthology.nuke:targets:
2021-02-10T14:20:41.068 INFO:teuthology.kill:  smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFvzppYYR57edZaDkupbBQi5Ieok0uzMsuCNsuduwCMDTIusIYDzReWC2UF4kvgQtbBs0eiA79B0tcnbRyxriqvtqHyqQEm446ULYnTQybDFGbrXVOkykAC7YfMhvKf+/0uca3YQlWqxiiQCy0KyGEbU1qf8dvytuwZo7htJoQxC88POBaP5kfascNzCKvGREX61Mj7IbhbTNRY9ZdCmp640XUBcXFW8IrkbVm3YdyXDoh7AtxBQlOM2AOGZw7l+qruGqikwFholP8TbNYkuCMGvitYJPOIYLqbqSM8dcvVzTvVQRRhMPlFkv55UTnTnzpcsUw3Ou0KjZ7rZPioJohXLVvqtfGASolWeZVhWbG6AdM1UrkctuHv5TpE76EEkaWLA/fGcuXf6hZbP1QQ/Y53BQ82v5T9Y0M566jPJgirW5NIiYIY2vRoUAn2E2XH8X6BjTF3bGgdhrUbcoXTKilIiD/oCF2gFlB4v8mhsTgUFi7FtIpbKPeT7kr9gDBlsM=
2021-02-10T14:20:41.068 INFO:teuthology.kill:  smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0C5afvlTDOBTWnXC4MM0xsTyu8W8UNhjjMbtJmGwmSPWAJlJgUAeRCgBOlGYjZN2X8a+9fEpyIy2NCPXxLu1LP+hBifs6BQ/z/09SMpheBMJiKeBgx+hvviYCdFyjw+VRhDWOUKfhqCfgh6gzvvRm1YP/gaRDuiqN+Rr9rnO17YcNGRhclSFVjeiQkDOcO4mer4OowNPGQD8C++3qx3BjQWhAHj1fxTPtdlks4KZr9tOPjYhgBkEmOhdKOdZ7EhXIfhifCs2wiMocWL7RfhhWqV30KsC2HtdT2rRfA7F2A9XW4jffjCNfJz5pzCNpIVnSXI2OLmGuLNi3s9uX0HjMcHPg/aSgMg116JGTenY2fs4FObuVm22dnVQ1CJ7kqhilK13PfO8b/sjjOnBJQHUDY2Ouw/tMjL6BYvw5evf9bRy+3POjj43CM4azAbvVNH/PzyNZrlVoByXqACSpk+4G2Wb9Xv0b744dPMBzli/iVnjlyBkQRwB2MA/oIbQqbtc=
2021-02-10T14:20:41.100 INFO:teuthology.kill:2021-02-10 14:20:41,099.099 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-10T14:20:41.103 INFO:teuthology.kill:2021-02-10 14:20:41,103.103 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-10T14:20:41.171 INFO:teuthology.kill:2021-02-10 14:20:41,170.170 INFO:teuthology.orchestra.console:Power off smithi155
2021-02-10T14:20:41.189 INFO:teuthology.kill:2021-02-10 14:20:41,189.189 INFO:teuthology.orchestra.console:Power off smithi116
2021-02-10T14:20:49.574 INFO:teuthology.kill:2021-02-10 14:20:49,574.574 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2021-02-10T14:20:49.724 INFO:teuthology.kill:2021-02-10 14:20:49,723.723 INFO:teuthology.lock.ops:unlocked smithi116.front.sepia.ceph.com
2021-02-10T14:20:53.694 INFO:teuthology.kill:2021-02-10 14:20:53,694.694 INFO:teuthology.orchestra.console:Power off for smithi155 completed
2021-02-10T14:20:53.836 INFO:teuthology.kill:2021-02-10 14:20:53,835.835 INFO:teuthology.lock.ops:unlocked smithi155.front.sepia.ceph.com
2021-02-10T14:22:54.048 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-02-10T14:22:54.189 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-02-10T14:22:54.189 INFO:teuthology.nuke:Checking targets against current locks
2021-02-10T14:22:54.295 INFO:teuthology.nuke:Not nuking smithi155.front.sepia.ceph.com because description doesn't match
2021-02-10T14:22:54.296 INFO:teuthology.nuke:Not nuking smithi116.front.sepia.ceph.com because description doesn't match