2021-11-26T23:20:45.910 INFO:root:teuthology version: 1.1.0-a0bc3188
2021-11-26T23:20:46.073 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi040.front.sepia.ceph.com' reimaging
2021-11-26T23:20:46.074 INFO:teuthology.lock.ops:Updating [ubuntu@smithi040.front.sepia.ceph.com]: reset os type and version on server
2021-11-26T23:20:46.076 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2021-11-26T23:20:46.098 INFO:teuthology.lock.ops:Node 'ubuntu@smithi040.front.sepia.ceph.com' reimaging is complete
2021-11-26T23:20:46.098 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi168.front.sepia.ceph.com' reimaging
2021-11-26T23:20:46.098 INFO:teuthology.lock.ops:Updating [ubuntu@smithi168.front.sepia.ceph.com]: reset os type and version on server
2021-11-26T23:20:46.099 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server
2021-11-26T23:20:46.120 INFO:teuthology.lock.ops:Node 'ubuntu@smithi168.front.sepia.ceph.com' reimaging is complete
2021-11-26T23:20:46.384 INFO:teuthology.provision.fog.smithi040:Scheduling deploy of centos 8.stream
2021-11-26T23:20:46.389 INFO:teuthology.provision.fog.smithi168:Scheduling deploy of centos 8.stream
2021-11-26T23:20:46.836 INFO:teuthology.orchestra.console:Power off smithi168
2021-11-26T23:20:46.887 INFO:teuthology.orchestra.console:Power off smithi040
2021-11-26T23:20:55.230 INFO:teuthology.orchestra.console:Power off for smithi168 completed
2021-11-26T23:20:55.274 INFO:teuthology.orchestra.console:Power off for smithi040 completed
2021-11-26T23:20:55.331 INFO:teuthology.orchestra.console:Power on smithi168
2021-11-26T23:20:55.393 INFO:teuthology.orchestra.console:Power on smithi040
2021-11-26T23:21:03.805 INFO:teuthology.orchestra.console:Power on for smithi040 completed
2021-11-26T23:21:03.853 INFO:teuthology.orchestra.console:Power on for smithi168 completed
2021-11-26T23:21:03.907 INFO:teuthology.provision.fog.smithi040:Waiting for deploy to finish
2021-11-26T23:21:03.954 INFO:teuthology.provision.fog.smithi168:Waiting for deploy to finish
2021-11-26T23:25:38.765 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-26T23:25:39.060 INFO:teuthology.provision.fog.smithi040:Node is ready
2021-11-26T23:25:39.079 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040.front.sepia.ceph.com
2021-11-26T23:25:39.137 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40 smithi040.front.sepia.ceph.com smithi040
2021-11-26T23:25:39.491 INFO:teuthology.provision.fog.smithi040:Deploy complete!
2021-11-26T23:26:09.101 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-26T23:26:09.378 INFO:teuthology.provision.fog.smithi168:Node is ready
2021-11-26T23:26:09.394 INFO:teuthology.orchestra.run.smithi168.stdout:smithi168.front.sepia.ceph.com
2021-11-26T23:26:09.451 INFO:teuthology.orchestra.run.smithi168.stdout:172.21.15.168 smithi168.front.sepia.ceph.com smithi168
2021-11-26T23:26:09.790 INFO:teuthology.provision.fog.smithi168:Deploy complete!
2021-11-26T23:26:09.951 INFO:teuthology.lock.ops:Checking smithi040.front.sepia.ceph.com
2021-11-26T23:26:09.952 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-26T23:26:09.971 INFO:teuthology.lock.ops:Checking smithi168.front.sepia.ceph.com
2021-11-26T23:26:09.972 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-26T23:26:09.991 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: set os type and version on server
2021-11-26T23:26:09.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com'
2021-11-26T23:26:10.233 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64
2021-11-26T23:26:10.289 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream"
2021-11-26T23:26:10.290 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="8"
2021-11-26T23:26:10.290 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos"
2021-11-26T23:26:10.290 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora"
2021-11-26T23:26:10.291 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="8"
2021-11-26T23:26:10.291 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el8"
2021-11-26T23:26:10.291 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-26T23:26:10.291 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31"
2021-11-26T23:26:10.291 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-26T23:26:10.292 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/"
2021-11-26T23:26:10.292 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-26T23:26:10.292 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-26T23:26:10.292 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-26T23:26:10.294 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2021-11-26T23:26:10.312 INFO:teuthology.lock.ops:Updating [smithi168.front.sepia.ceph.com]: set os type and version on server
2021-11-26T23:26:10.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi168.front.sepia.ceph.com'
2021-11-26T23:26:10.502 INFO:teuthology.orchestra.run.smithi168.stdout:x86_64
2021-11-26T23:26:10.558 INFO:teuthology.orchestra.run.smithi168.stdout:NAME="CentOS Stream"
2021-11-26T23:26:10.559 INFO:teuthology.orchestra.run.smithi168.stdout:VERSION="8"
2021-11-26T23:26:10.559 INFO:teuthology.orchestra.run.smithi168.stdout:ID="centos"
2021-11-26T23:26:10.559 INFO:teuthology.orchestra.run.smithi168.stdout:ID_LIKE="rhel fedora"
2021-11-26T23:26:10.559 INFO:teuthology.orchestra.run.smithi168.stdout:VERSION_ID="8"
2021-11-26T23:26:10.560 INFO:teuthology.orchestra.run.smithi168.stdout:PLATFORM_ID="platform:el8"
2021-11-26T23:26:10.560 INFO:teuthology.orchestra.run.smithi168.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-26T23:26:10.560 INFO:teuthology.orchestra.run.smithi168.stdout:ANSI_COLOR="0;31"
2021-11-26T23:26:10.560 INFO:teuthology.orchestra.run.smithi168.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-26T23:26:10.560 INFO:teuthology.orchestra.run.smithi168.stdout:HOME_URL="https://centos.org/"
2021-11-26T23:26:10.561 INFO:teuthology.orchestra.run.smithi168.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-26T23:26:10.561 INFO:teuthology.orchestra.run.smithi168.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-26T23:26:10.561 INFO:teuthology.orchestra.run.smithi168.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-26T23:26:10.562 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server
2021-11-26T23:26:10.679 INFO:teuthology.dispatcher.supervisor:Running job 6527157
2021-11-26T23:26:10.680 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a0bc318848afe71da1fbf34e1e1b44738d8e3d6d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-25_15:23:56-rados-wip-yuriw-master-11.24.21-distro-basic-smithi/6527157 --name yuriw-2021-11-25_15:23:56-rados-wip-yuriw-master-11.24.21-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2021-11-25_15:23:56-rados-wip-yuriw-master-11.24.21-distro-basic-smithi/6527157/orig.config.yaml
2021-11-26T23:26:10.683 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-25_15:23:56-rados-wip-yuriw-master-11.24.21-distro-basic-smithi/6527157
2021-11-26T23:26:10.684 INFO:teuthology.dispatcher.supervisor:Job PID: 7911
2021-11-26T23:26:10.684 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-27T11:27:05.562 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-27T11:27:05.814 INFO:teuthology.kill:Killing Pids: {7911}
2021-11-27T11:27:06.990 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-27T11:27:06.991 INFO:teuthology.task.internal:roles: ubuntu@smithi168.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-27T11:27:06.992 INFO:teuthology.misc:Compressing logs...
2021-11-27T11:27:06.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com'
2021-11-27T11:27:07.187 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi168.front.sepia.ceph.com'
2021-11-27T11:27:07.485 INFO:teuthology.misc:Compressing logs...
2021-11-27T11:27:08.804 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-11-27T11:27:09.209 INFO:teuthology.misc:Compressing logs...
2021-11-27T11:27:28.328 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-11-27T11:27:30.285 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-mon.smithi040.log: file size changed while zipping
2021-11-27T11:27:44.326 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-11-27T11:27:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-11-27T11:28:01.716 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-11-27T11:28:01.817 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-mds.cephfs.smithi040.kidmle.log: file size changed while zipping
2021-11-27T11:28:19.885 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-11-27T11:28:28.029 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-mgr.smithi040.oraggl.log: file size changed while zipping
2021-11-27T11:28:33.400 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-mds.cephfs.smithi168.htfeqm.log: file size changed while zipping
2021-11-27T11:29:28.266 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-mds.cephfs.smithi168.bzrllv.log: file size changed while zipping
2021-11-27T11:29:36.538 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-mon.smithi168.log: file size changed while zipping
2021-11-27T11:29:50.975 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/10a92e82-4f12-11ec-8c2d-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-11-27T11:30:40.300 INFO:teuthology.kill:No teuthology processes running
2021-11-27T11:30:41.508 INFO:teuthology.kill:Nuking machines: ['smithi040', 'smithi168']
2021-11-27T11:30:42.506 INFO:teuthology.kill:2021-11-27 11:30:42,505.505 INFO:teuthology.nuke:targets:
2021-11-27T11:30:42.507 INFO:teuthology.kill:  smithi040.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3CCK6ipnRhF/68D3J1hYyAf3qcfbSSOwP+m9HU2V4JIxxG+kPDsXmbSQT1VSPa1BOFCoFNISWBE/OM9gLziYOFazYr5eJ5Wi5+dDOLvu9VKVSW+nNeqE55etkzgYgykbZg42d6oBwXg4qvmHkiJwejfOQRhYl0bzVOkS+SgMHpdYvWSCs925KPHQGArCrlXdoSQGC5oJG1bY7zKdFqdjcc3MJsKKBx7DkkUt6/Ldm19FMeO2eDPjUTZYZZG5Qk079EsLPrLNTLDxGpzPnx31z0cYof5QY/D4KjOBqTdUwGQICDcbDoQ9Di5YTtIQ0utZ/K7EhWqVA/payJTeGZ3wrkuIrH9IOnMuDaJAyrzLDX/+PgUiIdWpj3QdEyL2LSnQ7P9CPBhZPzGMxOKWYpFHnShDPF2YgUD9UqKu+7rKiMgcBihCpBKg+HiotyQTEfBQiNnEQdSHKV5cANB+DL/P0h6oAiSAhToqsD3oZ0hkP0JkXpFhrNLRB5b7dKJkzDTs=
2021-11-27T11:30:42.507 INFO:teuthology.kill:  smithi168.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoVGU4jGafzzojR11w7rwcJCYAeL5e1srH9qrgEJHXUJ7eNlrkDsc9ASEiVeJaEq8/Y1xhyq/mAiBHlnXtHDtpdGWM1u1Sm9VnlNBB6v9i9JeM7adym519scD62rIsnLPXWJOryQYKni1rW0fEEtyiAeZI9xVBK/OIG765b58bPlF1xKG4u/gxBKAz3pB3s3qPnPrkAQNXxJuQKzay3bMtr/D7V0pP3PZiSNqpbU7tlMLkkZHAKwRqRNAzzdaiuPwQqKHshK6AvZMHXbsgOrSXowXODkOqbw3Jh0tyVXMWKvNhcTNLoKY+tUOmxUWlE5SYPvW2ugYylqF/X6UCedpysdGh1nw/yETCR++Jlw2stUCyW0hCRIGvYFg80aJ04Eut10+hEQAw8fBQkcJYmWrHxFoLEykPC1P3JFtY62j0VIzR9t8vO+kxL3DWJMR19QcUiHiN20cGVKrhkSurPCHMjBfWWvNijwiPCyCvYFjXztK4PKpbJKuddejvQriy1JM=
2021-11-27T11:30:42.533 INFO:teuthology.kill:2021-11-27 11:30:42,533.533 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-27T11:30:42.537 INFO:teuthology.kill:2021-11-27 11:30:42,537.537 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-27T11:30:42.626 INFO:teuthology.kill:2021-11-27 11:30:42,624.624 INFO:teuthology.orchestra.console:Power off smithi040
2021-11-27T11:30:42.657 INFO:teuthology.kill:2021-11-27 11:30:42,656.656 INFO:teuthology.orchestra.console:Power off smithi168
2021-11-27T11:30:51.017 INFO:teuthology.kill:2021-11-27 11:30:51,016.016 INFO:teuthology.orchestra.console:Power off for smithi040 completed
2021-11-27T11:30:51.028 INFO:teuthology.kill:2021-11-27 11:30:51,028.028 INFO:teuthology.orchestra.console:Power off for smithi168 completed
2021-11-27T11:30:51.166 INFO:teuthology.kill:2021-11-27 11:30:51,165.165 INFO:teuthology.lock.ops:unlocked: smithi040.front.sepia.ceph.com
2021-11-27T11:30:51.169 INFO:teuthology.kill:2021-11-27 11:30:51,169.169 INFO:teuthology.lock.ops:unlocked: smithi168.front.sepia.ceph.com
2021-11-27T11:32:51.507 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-27T11:32:51.713 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi040 but it was locked by another job: /home/teuthworker/archive/ifed01-2021-11-26_20:37:20-rados-wip-ifed-daemonhistogram-distro-default-smithi/6529724
2021-11-27T11:32:51.713 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi168 but it was locked by another job: /home/teuthworker/archive/ifed01-2021-11-26_20:37:20-rados-wip-ifed-daemonhistogram-distro-default-smithi/6529724