2021-10-25T04:06:30.227 INFO:root:teuthology version: 1.1.0-c56135d1
2021-10-25T04:06:30.350 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi045.front.sepia.ceph.com' reimaging
2021-10-25T04:06:30.351 INFO:teuthology.lock.ops:Updating [ubuntu@smithi045.front.sepia.ceph.com]: reset os type and version on server
2021-10-25T04:06:30.351 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server
2021-10-25T04:06:30.369 INFO:teuthology.lock.ops:Node 'ubuntu@smithi045.front.sepia.ceph.com' reimaging is complete
2021-10-25T04:06:30.369 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging
2021-10-25T04:06:30.370 INFO:teuthology.lock.ops:Updating [ubuntu@smithi146.front.sepia.ceph.com]: reset os type and version on server
2021-10-25T04:06:30.370 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2021-10-25T04:06:30.388 INFO:teuthology.lock.ops:Node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging is complete
2021-10-25T04:06:30.579 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of centos 8.3
2021-10-25T04:06:30.589 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 8.3
2021-10-25T04:06:30.984 INFO:teuthology.orchestra.console:Power off smithi045
2021-10-25T04:06:31.002 INFO:teuthology.orchestra.console:Power off smithi146
2021-10-25T04:06:39.363 INFO:teuthology.orchestra.console:Power off for smithi045 completed
2021-10-25T04:06:39.365 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2021-10-25T04:06:39.465 INFO:teuthology.orchestra.console:Power on smithi045
2021-10-25T04:06:39.472 INFO:teuthology.orchestra.console:Power on smithi146
2021-10-25T04:06:47.830 INFO:teuthology.orchestra.console:Power on for smithi045 completed
2021-10-25T04:06:47.836 INFO:teuthology.orchestra.console:Power on for smithi146 completed
2021-10-25T04:06:47.932 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish
2021-10-25T04:06:47.937 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2021-10-25T04:11:22.841 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-25T04:11:23.166 INFO:teuthology.provision.fog.smithi045:Node is ready
2021-10-25T04:11:23.182 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com
2021-10-25T04:11:23.240 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045
2021-10-25T04:11:23.573 INFO:teuthology.provision.fog.smithi045:Deploy complete!
2021-10-25T04:11:34.775 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-25T04:11:35.087 INFO:teuthology.provision.fog.smithi146:Node is ready
2021-10-25T04:11:35.102 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2021-10-25T04:11:35.158 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2021-10-25T04:11:35.477 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2021-10-25T04:11:35.624 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com
2021-10-25T04:11:35.625 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-25T04:11:35.646 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2021-10-25T04:11:35.646 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-25T04:11:35.670 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server
2021-10-25T04:11:35.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com'
2021-10-25T04:11:35.944 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64
2021-10-25T04:11:35.999 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Linux"
2021-10-25T04:11:36.000 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8"
2021-10-25T04:11:36.000 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos"
2021-10-25T04:11:36.000 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora"
2021-10-25T04:11:36.000 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8"
2021-10-25T04:11:36.001 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8"
2021-10-25T04:11:36.001 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Linux 8"
2021-10-25T04:11:36.001 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31"
2021-10-25T04:11:36.001 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-10-25T04:11:36.002 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/"
2021-10-25T04:11:36.002 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-10-25T04:11:36.002 INFO:teuthology.orchestra.run.smithi045.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-10-25T04:11:36.002 INFO:teuthology.orchestra.run.smithi045.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-10-25T04:11:36.004 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server
2021-10-25T04:11:36.032 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2021-10-25T04:11:36.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2021-10-25T04:11:36.290 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2021-10-25T04:11:36.344 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Linux"
2021-10-25T04:11:36.344 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8"
2021-10-25T04:11:36.345 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2021-10-25T04:11:36.345 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2021-10-25T04:11:36.345 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8"
2021-10-25T04:11:36.345 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8"
2021-10-25T04:11:36.345 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Linux 8"
2021-10-25T04:11:36.346 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2021-10-25T04:11:36.346 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-10-25T04:11:36.346 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2021-10-25T04:11:36.346 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-10-25T04:11:36.347 INFO:teuthology.orchestra.run.smithi146.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-10-25T04:11:36.347 INFO:teuthology.orchestra.run.smithi146.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-10-25T04:11:36.347 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2021-10-25T04:11:36.489 INFO:teuthology.dispatcher.supervisor:Running job 6459895
2021-10-25T04:11:36.490 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-10-25_03:06:01-fs-wip-pdonnell-testing-20211025.000447-distro-basic-smithi/6459895 --name pdonnell-2021-10-25_03:06:01-fs-wip-pdonnell-testing-20211025.000447-distro-basic-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/pdonnell-2021-10-25_03:06:01-fs-wip-pdonnell-testing-20211025.000447-distro-basic-smithi/6459895/orig.config.yaml
2021-10-25T04:11:36.493 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-10-25_03:06:01-fs-wip-pdonnell-testing-20211025.000447-distro-basic-smithi/6459895
2021-10-25T04:11:36.494 INFO:teuthology.dispatcher.supervisor:Job PID: 14860
2021-10-25T04:11:36.494 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-10-25T16:12:27.416 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-10-25T16:12:27.617 INFO:teuthology.kill:Killing Pids: {14860}
2021-10-25T16:12:28.739 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-10-25T16:12:28.740 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-10-25T16:12:28.741 INFO:teuthology.misc:Compressing logs...
2021-10-25T16:12:28.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com'
2021-10-25T16:12:28.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2021-10-25T16:12:29.246 INFO:teuthology.misc:Compressing logs...
2021-10-25T16:12:29.595 INFO:teuthology.misc:Compressing logs...
2021-10-25T16:12:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-mgr.smithi045.wsnmpf.log: file size changed while zipping
2021-10-25T16:12:46.378 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-10-25T16:12:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-10-25T16:13:04.477 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-10-25T16:13:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-mon.smithi045.log: file size changed while zipping
2021-10-25T16:13:31.198 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-mds.cephfs.smithi146.dyggbv.log: file size changed while zipping
2021-10-25T16:13:38.452 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-mon.smithi146.log: file size changed while zipping
2021-10-25T16:13:38.741 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-10-25T16:13:51.636 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-10-25T16:13:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-10-25T16:14:12.325 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/6ac00c50-354b-11ec-8c28-001a4aab830c/ceph-mds.cephfs.smithi045.itxslm.log: file size changed while zipping
2021-10-25T16:14:50.203 INFO:teuthology.kill:No teuthology processes running
2021-10-25T16:14:51.260 INFO:teuthology.kill:Nuking machines: ['smithi045', 'smithi146']
2021-10-25T16:14:52.206 INFO:teuthology.kill:2021-10-25 16:14:52,205.205 INFO:teuthology.nuke:targets:
2021-10-25T16:14:52.207 INFO:teuthology.kill:  smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZE0fMKHqL3xhT/2V0C5FNpj+7ySsCr214V4Sx1TmrTkKzoOfkjOSCaocdtayYtqgDEzOq1ouhP4dWbympdwIRjAnk+Y+TGeVEOT5fR22FTdIbJhBHhm95khCK5eVijhA+w+hsG+jEaYQZZhRI+HjDpT+Mwr44A3q9GzRWMN0gO3QGOsWmjTa+W6z9YnoGblf2mq5nCGL2TA9fzsljF7isWSiRWgtCG6DsBj4pTvLPZqfkxrjG1135eir23qfov9VtDLI3Lr/PXFsFIIZnGjeWLcn9gkOOIKYYrG0MHYJde+0d0oz2ndEXE/5V9hENq5i61XEPvUMGwNSdtUc7WUepE+qDKYZI9y1Y04QM5IgOZIN493ZAoM0N9ImYC/nGERo588InR3WJsdvUDdXhUzE5ANQSknjCg/Ngi85Oh4kbE3cjzMmN+KvROmLg8fFgP5a26ZLHPOqFvMu4apYmKZ1lXUbcdLvck5d70Y42/jilfvrlc/+zRYEq0ew6mD2u8jk=
2021-10-25T16:14:52.207 INFO:teuthology.kill:  smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSCGE3Tc1h2o8NLDVNsqD58NtO0dJxzP3AM6QDMvmqlgOPH0yDNbs4blE9oaOszV8zbp6IXQldmr/mvWYiHHaPqyQqR0QLvJXCtwpV521qR74izTGsOx2ErfaOXaK3JmJDvjutcmbz4F2czTm7PpWg+NjniFB7girhRdqbJU4s8ORJKn7koobqMEudkhYJGnSjkYcIpa9hpBKuxv05j61oP/KMS2NoYtxpcPR/3AueMp/fM1NXy5witAci3xlCdfeLikZ149TKtQeVQhq26BADYO18x3OI13X48qG4UYUGbcDxzd+wAgb8fPikOPpUkkpECELX2b0s+F3ccC/Dz0MoWusg9kZ9t+YeO6KJ2+2nDhIN70xbr8hnADhYfgO55nMNFjvxd9LTBvfg8rC92ZLZdWpdoGVRHG+LSCaE+P/D654f9z+ddLBRLsI6N1i9F3oUPF4S4r985qr34ZlmA9cl7Eumd8tBrC37/HywZ5BF+YVG7W1renoR/P89ZceN7TE=
2021-10-25T16:14:52.233 INFO:teuthology.kill:2021-10-25 16:14:52,233.233 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-25T16:14:52.237 INFO:teuthology.kill:2021-10-25 16:14:52,236.236 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-25T16:14:52.311 INFO:teuthology.kill:2021-10-25 16:14:52,311.311 INFO:teuthology.orchestra.console:Power off smithi045
2021-10-25T16:14:52.324 INFO:teuthology.kill:2021-10-25 16:14:52,323.323 INFO:teuthology.orchestra.console:Power off smithi146
2021-10-25T16:15:00.687 INFO:teuthology.kill:2021-10-25 16:15:00,687.687 INFO:teuthology.orchestra.console:Power off for smithi045 completed
2021-10-25T16:15:00.835 INFO:teuthology.kill:2021-10-25 16:15:00,834.834 INFO:teuthology.lock.ops:unlocked: smithi045.front.sepia.ceph.com
2021-10-25T16:15:04.807 INFO:teuthology.kill:2021-10-25 16:15:04,806.806 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2021-10-25T16:15:04.952 INFO:teuthology.kill:2021-10-25 16:15:04,951.951 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com
2021-10-25T16:17:05.248 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15