2021-12-15T18:52:52.419 INFO:root:teuthology version: 1.1.0-95a7d479
2021-12-15T18:52:52.545 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi063.front.sepia.ceph.com' reimaging
2021-12-15T18:52:52.546 INFO:teuthology.lock.ops:Updating [ubuntu@smithi063.front.sepia.ceph.com]: reset os type and version on server
2021-12-15T18:52:52.546 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server
2021-12-15T18:52:52.564 INFO:teuthology.lock.ops:Node 'ubuntu@smithi063.front.sepia.ceph.com' reimaging is complete
2021-12-15T18:52:52.564 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi158.front.sepia.ceph.com' reimaging
2021-12-15T18:52:52.564 INFO:teuthology.lock.ops:Updating [ubuntu@smithi158.front.sepia.ceph.com]: reset os type and version on server
2021-12-15T18:52:52.565 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2021-12-15T18:52:52.581 INFO:teuthology.lock.ops:Node 'ubuntu@smithi158.front.sepia.ceph.com' reimaging is complete
2021-12-15T18:52:52.763 INFO:teuthology.provision.fog.smithi063:Scheduling deploy of centos 8.3
2021-12-15T18:52:52.795 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of centos 8.3
2021-12-15T18:52:53.110 INFO:teuthology.orchestra.console:Power off smithi063
2021-12-15T18:52:53.127 INFO:teuthology.orchestra.console:Power off smithi158
2021-12-15T18:53:01.494 INFO:teuthology.orchestra.console:Power off for smithi158 completed
2021-12-15T18:53:01.595 INFO:teuthology.orchestra.console:Power on smithi158
2021-12-15T18:53:05.598 INFO:teuthology.orchestra.console:Power off for smithi063 completed
2021-12-15T18:53:05.700 INFO:teuthology.orchestra.console:Power on smithi063
2021-12-15T18:53:09.953 INFO:teuthology.orchestra.console:Power on for smithi063 completed
2021-12-15T18:53:09.957 INFO:teuthology.orchestra.console:Power on for smithi158 completed
2021-12-15T18:53:10.054 INFO:teuthology.provision.fog.smithi063:Waiting for deploy to finish
2021-12-15T18:53:10.060 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish
2021-12-15T18:58:15.133 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-15T18:58:15.454 INFO:teuthology.provision.fog.smithi063:Node is ready
2021-12-15T18:58:15.469 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063.front.sepia.ceph.com
2021-12-15T18:58:15.524 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63 smithi063.front.sepia.ceph.com smithi063
2021-12-15T18:58:15.871 INFO:teuthology.provision.fog.smithi063:Deploy complete!
2021-12-15T18:58:30.169 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-15T18:58:30.433 INFO:teuthology.provision.fog.smithi158:Node is ready
2021-12-15T18:58:30.447 INFO:teuthology.orchestra.run.smithi158.stdout:smithi158.front.sepia.ceph.com
2021-12-15T18:58:30.503 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.158 smithi158.front.sepia.ceph.com smithi158
2021-12-15T18:58:30.828 INFO:teuthology.provision.fog.smithi158:Deploy complete!
2021-12-15T18:58:30.982 INFO:teuthology.lock.ops:Checking smithi063.front.sepia.ceph.com
2021-12-15T18:58:30.983 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-15T18:58:31.002 INFO:teuthology.lock.ops:Checking smithi158.front.sepia.ceph.com
2021-12-15T18:58:31.003 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-15T18:58:31.023 INFO:teuthology.lock.ops:Updating [smithi063.front.sepia.ceph.com]: set os type and version on server
2021-12-15T18:58:31.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com'
2021-12-15T18:58:31.301 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64
2021-12-15T18:58:31.356 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Linux"
2021-12-15T18:58:31.357 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="8"
2021-12-15T18:58:31.357 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos"
2021-12-15T18:58:31.357 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora"
2021-12-15T18:58:31.357 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="8"
2021-12-15T18:58:31.357 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el8"
2021-12-15T18:58:31.358 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Linux 8"
2021-12-15T18:58:31.358 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31"
2021-12-15T18:58:31.358 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-15T18:58:31.358 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/"
2021-12-15T18:58:31.359 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-12-15T18:58:31.359 INFO:teuthology.orchestra.run.smithi063.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-12-15T18:58:31.359 INFO:teuthology.orchestra.run.smithi063.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-12-15T18:58:31.360 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server
2021-12-15T18:58:31.383 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: set os type and version on server
2021-12-15T18:58:31.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2021-12-15T18:58:31.584 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64
2021-12-15T18:58:31.639 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Linux"
2021-12-15T18:58:31.640 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8"
2021-12-15T18:58:31.640 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos"
2021-12-15T18:58:31.640 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora"
2021-12-15T18:58:31.641 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8"
2021-12-15T18:58:31.641 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8"
2021-12-15T18:58:31.641 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Linux 8"
2021-12-15T18:58:31.641 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31"
2021-12-15T18:58:31.641 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-15T18:58:31.642 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/"
2021-12-15T18:58:31.642 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-12-15T18:58:31.642 INFO:teuthology.orchestra.run.smithi158.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-12-15T18:58:31.642 INFO:teuthology.orchestra.run.smithi158.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-12-15T18:58:31.643 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2021-12-15T18:58:31.736 INFO:teuthology.dispatcher.supervisor:Running job 6564519
2021-12-15T18:58:31.737 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-12-15_15:59:44-rados-wip-sage2-testing-2021-12-14-1714-distro-basic-smithi/6564519 --name sage-2021-12-15_15:59:44-rados-wip-sage2-testing-2021-12-14-1714-distro-basic-smithi --description rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} -- /home/teuthworker/archive/sage-2021-12-15_15:59:44-rados-wip-sage2-testing-2021-12-14-1714-distro-basic-smithi/6564519/orig.config.yaml
2021-12-15T18:58:31.740 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-12-15_15:59:44-rados-wip-sage2-testing-2021-12-14-1714-distro-basic-smithi/6564519
2021-12-15T18:58:31.741 INFO:teuthology.dispatcher.supervisor:Job PID: 23828
2021-12-15T18:58:31.741 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-16T06:59:27.180 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-16T06:59:27.360 INFO:teuthology.kill:Killing Pids: {23828}
2021-12-16T06:59:28.595 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a']
2021-12-16T06:59:28.596 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2021-12-16T06:59:28.596 INFO:teuthology.misc:Compressing logs...
2021-12-16T06:59:28.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com'
2021-12-16T06:59:28.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2021-12-16T06:59:29.139 INFO:teuthology.misc:Compressing logs...
2021-12-16T06:59:32.475 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-16T06:59:33.577 INFO:teuthology.misc:Compressing logs...
2021-12-16T06:59:45.268 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-16T06:59:58.271 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-16T07:00:00.972 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-16T07:00:05.687 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-mgr.x.log: file size changed while zipping
2021-12-16T07:00:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-16T07:00:24.860 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2021-12-16T07:00:27.560 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-16T07:00:31.277 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2021-12-16T07:00:34.091 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-16T07:00:44.126 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-16T07:00:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/bfd5c218-5dda-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-16T07:02:33.971 INFO:teuthology.orchestra.run.smithi158.stderr:tar: ./bfd5c218-5dda-11ec-8c31-001a4aab830c: file changed as we read it
2021-12-16T07:02:34.286 INFO:teuthology.kill:No teuthology processes running
2021-12-16T07:02:35.458 INFO:teuthology.kill:Nuking machines: ['smithi063', 'smithi158']
2021-12-16T07:02:36.464 INFO:teuthology.kill:2021-12-16 07:02:36,463.463 INFO:teuthology.nuke:targets:
2021-12-16T07:02:36.465 INFO:teuthology.kill:  smithi063.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdizhw9H2zoIvwIVkfVm7zDSKskINqb5EaMrgR0/7lK80KpJBZ1FDW+pnmSPHXkYTYOGgMv4vEIciGQ4tkL4XlyPbbArmAD3lWErZPYeN3LbJuRLLKnLty7IBhhU6kPVGOlLRYY5gPRrgvHKR0L6bQoriW9Pt/Gu5/AZNB3k131SVv1B3S0/1kjmrXca7Wzwpdt5JiGe9tYdKGPUmjjFtKCH8U7QXKR2PSHa89linyKqVrCDvHz19KStrzxXaN6Dq4NYt8CCQ/od7LSrUvCR5WPQ0ueNxl96EJaqDLPM1bbA02nlgSuTDo/t9Sq0qombBkpD8MHCHF7qrNSf/Pe+vfYXfGhavvaK2o6cUuGSYfAg943e7IUCUa+sgYibHbv1KkT9D/xxOJNXKNF9x91PAGsj/PW81MGWauG/diiMX9q8nfcLkOUrMrFUMsNyheL4Xl+qMYEfTeRNZLcr6sXuMYz1VxD0y5m/3pfct/Fo9yPm8oleQLv+Hzar1M81QSiVE=
2021-12-16T07:02:36.465 INFO:teuthology.kill:  smithi158.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLhDnNEzoQDbgVbTn1egZIF4bqtKBP9W7Ttyl401tUPoXznFRmd/f9hREeqnBn+Oal5lvyv/icb/RUNcFKj7MENkNooDLSsMegladnvvV2g9rR/Etq8RHVF1sEUbeQcQtVc+dXaA1BhB2XZtjEFMNhB/kILyB+0qtlI9lLeNf0/4l3Imh+mkBAmEq7D2QZCVzm4Vl/pM82opkDRxsyAlZo4e/GT/VAbKVwmDzqf6m/sL45FRN/8AR1cuSbOfVK3jLgZh79T9mmFfOfGKH0BiUiqqtazLFWAjlVw/VEV6YnWjg504SMNfW4ZxN6CvluTsMsxUa9rTIq8FJSLlxid1/FFTKyZc7xawxApqq65gayOnn8tfwXpyDpC9Df2aCMo2T136krtMx5X+ih93XPd91JzcRbL7IXM4xR8HzFR3CplzjCa+wNoo1wg3DKNcH/quVqzZ3de1vul+2sbfdxEH/OqrGh1OUsgGdcaHK7e2YE4WdRiTZ1jioaEpsXststKHk=
2021-12-16T07:02:36.491 INFO:teuthology.kill:2021-12-16 07:02:36,491.491 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-16T07:02:36.495 INFO:teuthology.kill:2021-12-16 07:02:36,494.494 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-16T07:02:36.569 INFO:teuthology.kill:2021-12-16 07:02:36,569.569 INFO:teuthology.orchestra.console:Power off smithi158
2021-12-16T07:02:36.584 INFO:teuthology.kill:2021-12-16 07:02:36,583.583 INFO:teuthology.orchestra.console:Power off smithi063
2021-12-16T07:02:49.057 INFO:teuthology.kill:2021-12-16 07:02:49,057.057 INFO:teuthology.orchestra.console:Power off for smithi158 completed
2021-12-16T07:02:49.065 INFO:teuthology.kill:2021-12-16 07:02:49,064.064 INFO:teuthology.orchestra.console:Power off for smithi063 completed
2021-12-16T07:02:49.280 INFO:teuthology.kill:2021-12-16 07:02:49,279.279 INFO:teuthology.lock.ops:unlocked: smithi158.front.sepia.ceph.com
2021-12-16T07:02:49.284 INFO:teuthology.kill:2021-12-16 07:02:49,284.284 INFO:teuthology.lock.ops:unlocked: smithi063.front.sepia.ceph.com
2021-12-16T07:04:49.646 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-16T07:04:49.740 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi063 but it was locked by another job: /home/teuthworker/archive/akupczyk-2021-12-15_18:10:21-rados-wip-bluefs-fine-grain-locking-4-rebase-distro-default-smithi/6565296
2021-12-16T07:04:49.742 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi158 but it was locked by another job: /home/teuthworker/archive/akupczyk-2021-12-15_18:10:21-rados-wip-bluefs-fine-grain-locking-4-rebase-distro-default-smithi/6565296