2021-12-05T15:09:13.527 INFO:root:teuthology version: 1.1.0-844d1377
2021-12-05T15:09:13.644 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi132.front.sepia.ceph.com' reimaging
2021-12-05T15:09:13.645 INFO:teuthology.lock.ops:Updating [ubuntu@smithi132.front.sepia.ceph.com]: reset os type and version on server
2021-12-05T15:09:13.646 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server
2021-12-05T15:09:13.664 INFO:teuthology.lock.ops:Node 'ubuntu@smithi132.front.sepia.ceph.com' reimaging is complete
2021-12-05T15:09:13.664 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi191.front.sepia.ceph.com' reimaging
2021-12-05T15:09:13.665 INFO:teuthology.lock.ops:Updating [ubuntu@smithi191.front.sepia.ceph.com]: reset os type and version on server
2021-12-05T15:09:13.665 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2021-12-05T15:09:13.683 INFO:teuthology.lock.ops:Node 'ubuntu@smithi191.front.sepia.ceph.com' reimaging is complete
2021-12-05T15:09:14.861 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of centos 8.stream
2021-12-05T15:09:15.003 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of centos 8.stream
2021-12-05T15:09:20.950 INFO:teuthology.orchestra.console:Power off smithi191
2021-12-05T15:09:21.091 INFO:teuthology.orchestra.console:Power off smithi132
2021-12-05T15:09:21.231 INFO:teuthology.orchestra.console:Power off for smithi132 completed
2021-12-05T15:09:21.332 INFO:teuthology.orchestra.console:Power on smithi132
2021-12-05T15:09:29.315 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2021-12-05T15:09:29.416 INFO:teuthology.orchestra.console:Power on smithi191
2021-12-05T15:09:29.686 INFO:teuthology.orchestra.console:Power on for smithi132 completed
2021-12-05T15:09:29.787 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish
2021-12-05T15:09:37.879 INFO:teuthology.orchestra.console:Power on for smithi191 completed
2021-12-05T15:09:37.980 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish
2021-12-05T15:16:21.098 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-05T15:16:21.383 INFO:teuthology.provision.fog.smithi132:Node is ready
2021-12-05T15:16:21.400 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com
2021-12-05T15:16:21.456 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132
2021-12-05T15:16:21.787 INFO:teuthology.provision.fog.smithi132:Deploy complete!
2021-12-05T15:16:25.206 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-05T15:16:25.474 INFO:teuthology.provision.fog.smithi191:Node is ready
2021-12-05T15:16:25.491 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191
2021-12-05T15:16:25.549 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191
2021-12-05T15:16:25.885 INFO:teuthology.provision.fog.smithi191:Deploy complete!
2021-12-05T15:16:26.027 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com
2021-12-05T15:16:26.028 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-05T15:16:26.045 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com
2021-12-05T15:16:26.046 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-05T15:16:26.063 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server
2021-12-05T15:16:26.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2021-12-05T15:16:26.256 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64
2021-12-05T15:16:26.313 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream"
2021-12-05T15:16:26.313 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8"
2021-12-05T15:16:26.313 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos"
2021-12-05T15:16:26.314 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora"
2021-12-05T15:16:26.314 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8"
2021-12-05T15:16:26.314 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8"
2021-12-05T15:16:26.314 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-05T15:16:26.315 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31"
2021-12-05T15:16:26.315 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-05T15:16:26.315 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/"
2021-12-05T15:16:26.315 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-05T15:16:26.316 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-05T15:16:26.316 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-05T15:16:26.317 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server
2021-12-05T15:16:26.334 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server
2021-12-05T15:16:26.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com'
2021-12-05T15:16:26.517 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64
2021-12-05T15:16:26.572 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream"
2021-12-05T15:16:26.573 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8"
2021-12-05T15:16:26.573 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos"
2021-12-05T15:16:26.573 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora"
2021-12-05T15:16:26.573 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8"
2021-12-05T15:16:26.574 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8"
2021-12-05T15:16:26.574 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-05T15:16:26.575 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31"
2021-12-05T15:16:26.575 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-05T15:16:26.575 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/"
2021-12-05T15:16:26.575 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-05T15:16:26.575 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-05T15:16:26.576 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-05T15:16:26.576 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2021-12-05T15:16:26.683 INFO:teuthology.dispatcher.supervisor:Running job 6546579
2021-12-05T15:16:26.684 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546579 --name benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546579/orig.config.yaml
2021-12-05T15:16:26.687 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546579
2021-12-05T15:16:26.688 INFO:teuthology.dispatcher.supervisor:Job PID: 26060
2021-12-05T15:16:26.688 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-06T03:17:20.706 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-06T03:17:21.105 INFO:teuthology.kill:Killing Pids: {26060}
2021-12-06T03:17:22.306 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-12-06T03:17:22.307 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-12-06T03:17:22.307 INFO:teuthology.misc:Compressing logs...
2021-12-06T03:17:22.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com'
2021-12-06T03:17:22.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com'
2021-12-06T03:17:22.776 INFO:teuthology.misc:Compressing logs...
2021-12-06T03:17:24.031 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-06T03:17:24.451 INFO:teuthology.misc:Compressing logs...
2021-12-06T03:17:27.372 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-mds.cephfs.smithi132.wgucjz.log: file size changed while zipping
2021-12-06T03:17:33.729 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-mon.smithi191.log: file size changed while zipping
2021-12-06T03:17:42.259 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-06T03:17:48.074 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-06T03:17:55.689 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-06T03:18:05.945 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-06T03:18:19.041 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-06T03:18:20.159 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-mon.smithi132.log: file size changed while zipping
2021-12-06T03:18:29.440 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-mds.cephfs.smithi191.htrjus.log: file size changed while zipping
2021-12-06T03:18:31.964 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-mgr.smithi132.luyvon.log: file size changed while zipping
2021-12-06T03:18:47.990 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-06T03:19:19.683 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/1cf7a4c4-55df-11ec-8c2e-001a4aab830c/ceph-mds.cephfs.smithi132.bnxalo.log: file size changed while zipping
2021-12-06T03:19:44.363 INFO:teuthology.orchestra.run.smithi132.stderr:tar: .: file changed as we read it
2021-12-06T03:19:57.517 INFO:teuthology.kill:No teuthology processes running
2021-12-06T03:19:58.750 INFO:teuthology.kill:Nuking machines: ['smithi132', 'smithi191']
2021-12-06T03:19:59.717 INFO:teuthology.kill:2021-12-06 03:19:59,716.716 INFO:teuthology.nuke:targets:
2021-12-06T03:19:59.718 INFO:teuthology.kill:  smithi132.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt33YHuPoc67aIYTTenl8O/vUpKUZBUtWAlngmvqcPHaGsGHEqXRAdcYzu5GLk73pvH2RZQcuUm7WpvEkZPJg5l8mLvXifLmAdVnc9yAY4a4ggLR/EFTuwdCPwj+gm6hssMs5IbCvbt6Xv+893JaCuNn0UKGyg88OhB791IYEqxF1YpU4n67k/uCZRU0gV4yG17RG+aSPVjMj/nse37Ytz8tCqaR6cwHSriTL1s7mbMDP8i9v3qfLahOKtcKBF/+v2gN5qERdfbQp8zjXlQOTKm5BXtLsZup/58wh2IuqsBY0USig3BgRe42PusekOzVTrZCpRMiRcs8qz8fwV4e9T+6G/sO0y/+ok0JJjWNaEpONslQgr7CZOawwctMVcntySJJhH4plpeYxs7OR8MBsAms4O7RMa7Yven4YHeMcYr1El4WHRchdj14uZGzZcyIff70wQ5sJpfPYno/Ukrbxx0SVrVt9rq1LhXLta8f2HmU7zYXeFmwnqCCdD5GcAK+M=
2021-12-06T03:19:59.718 INFO:teuthology.kill:  smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIE4Jqyo+ABeeg8cuC8XZk0oScf//hgCcHmxRkjADd1+dnoyhud3irMLo+bBsRDMaA+w3jIR2mC8st8Ads9z1oAtMs4rZJhz+f/wrRNecrr40JNbVjXGR0D1Djguvkp5+ljeOn86+6fL7E8WHH691KL053Je8OmdfDcTfAHxnwzT70WKQBaFI1AeuyqhjHeLksgQpg/l9tph/E7MnMscrZ6qR+bW2nGhIctovFZl7gEfDxD4TAAOeo6HPYGucQwry33+X0LeT23beNP7CclzyCCjlUjRS+gcLafhDf3Ug0qQIW5bSnID2kwNNV1GwyF0ukkoRg3De/FiZYgVYYwDcga4gUly6j4OnNRppOcfWMgUiy29tRJbSLQsYvzv1fYUsi5ZvoywmucOljj7avAgmizQk12sRCX/BW+jv0UK3ucAb1eFXNtSvUrXTvxG6VVmpukK9jmaEMmznQu6GtXZi/OOJai3dRve53ocnQOk75qWomaJA7Ac79lcSRMOkfRR0=
2021-12-06T03:19:59.745 INFO:teuthology.kill:2021-12-06 03:19:59,745.745 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-06T03:19:59.748 INFO:teuthology.kill:2021-12-06 03:19:59,748.748 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-06T03:19:59.826 INFO:teuthology.kill:2021-12-06 03:19:59,825.825 INFO:teuthology.orchestra.console:Power off smithi191
2021-12-06T03:19:59.839 INFO:teuthology.kill:2021-12-06 03:19:59,839.839 INFO:teuthology.orchestra.console:Power off smithi132
2021-12-06T03:20:08.198 INFO:teuthology.kill:2021-12-06 03:20:08,198.198 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2021-12-06T03:20:08.345 INFO:teuthology.kill:2021-12-06 03:20:08,344.344 INFO:teuthology.lock.ops:unlocked: smithi191.front.sepia.ceph.com
2021-12-06T03:20:12.314 INFO:teuthology.kill:2021-12-06 03:20:12,314.314 INFO:teuthology.orchestra.console:Power off for smithi132 completed
2021-12-06T03:20:12.460 INFO:teuthology.kill:2021-12-06 03:20:12,460.460 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com
2021-12-06T03:22:12.782 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-06T03:22:12.895 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi132 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_02:01:02-rbd-master-distro-default-smithi/6547064
2021-12-06T03:22:12.896 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi191 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_02:01:02-rbd-master-distro-default-smithi/6547064