2022-02-04T17:41:14.372 INFO:root:teuthology version: 1.1.0-3094160c
2022-02-04T17:41:14.502 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi078.front.sepia.ceph.com' reimaging
2022-02-04T17:41:14.503 INFO:teuthology.lock.ops:Updating [ubuntu@smithi078.front.sepia.ceph.com]: reset os type and version on server
2022-02-04T17:41:14.503 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server
2022-02-04T17:41:14.521 INFO:teuthology.lock.ops:Node 'ubuntu@smithi078.front.sepia.ceph.com' reimaging is complete
2022-02-04T17:41:14.521 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi141.front.sepia.ceph.com' reimaging
2022-02-04T17:41:14.522 INFO:teuthology.lock.ops:Updating [ubuntu@smithi141.front.sepia.ceph.com]: reset os type and version on server
2022-02-04T17:41:14.522 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server
2022-02-04T17:41:14.542 INFO:teuthology.lock.ops:Node 'ubuntu@smithi141.front.sepia.ceph.com' reimaging is complete
2022-02-04T17:41:14.716 INFO:teuthology.provision.fog.smithi078:Scheduling deploy of centos 8.stream
2022-02-04T17:41:14.748 INFO:teuthology.provision.fog.smithi141:Scheduling deploy of centos 8.stream
2022-02-04T17:41:15.010 INFO:teuthology.orchestra.console:Power off smithi078
2022-02-04T17:41:15.048 INFO:teuthology.orchestra.console:Power off smithi141
2022-02-04T17:41:23.409 INFO:teuthology.orchestra.console:Power off for smithi141 completed
2022-02-04T17:41:23.510 INFO:teuthology.orchestra.console:Power on smithi141
2022-02-04T17:41:27.491 INFO:teuthology.orchestra.console:Power off for smithi078 completed
2022-02-04T17:41:27.593 INFO:teuthology.orchestra.console:Power on smithi078
2022-02-04T17:41:31.844 INFO:teuthology.orchestra.console:Power on for smithi078 completed
2022-02-04T17:41:31.945 INFO:teuthology.provision.fog.smithi078:Waiting for deploy to finish
2022-02-04T17:41:31.965 INFO:teuthology.orchestra.console:Power on for smithi141 completed
2022-02-04T17:41:32.066 INFO:teuthology.provision.fog.smithi141:Waiting for deploy to finish
2022-02-04T17:44:03.196 ERROR:teuthology.orchestra.connection:Error authenticating with smithi141.front.sepia.ceph.com: Authentication failed.
2022-02-04T17:45:41.827 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-04T17:45:44.101 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-04T17:46:07.009 INFO:teuthology.provision.fog.smithi141:Node is ready
2022-02-04T17:46:07.031 INFO:teuthology.orchestra.run.smithi141.stdout:smithi141.front.sepia.ceph.com
2022-02-04T17:46:07.090 INFO:teuthology.orchestra.run.smithi141.stdout:172.21.15.141 smithi141.front.sepia.ceph.com smithi141
2022-02-04T17:46:07.472 INFO:teuthology.provision.fog.smithi141:Deploy complete!
2022-02-04T17:46:09.665 INFO:teuthology.provision.fog.smithi078:Node is ready
2022-02-04T17:46:09.687 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078.front.sepia.ceph.com
2022-02-04T17:46:09.745 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78 smithi078.front.sepia.ceph.com smithi078
2022-02-04T17:46:10.123 INFO:teuthology.provision.fog.smithi078:Deploy complete!
2022-02-04T17:46:10.303 INFO:teuthology.lock.ops:Checking smithi078.front.sepia.ceph.com
2022-02-04T17:46:10.304 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-04T17:46:10.324 INFO:teuthology.lock.ops:Checking smithi141.front.sepia.ceph.com
2022-02-04T17:46:10.324 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-04T17:46:10.343 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: set os type and version on server
2022-02-04T17:46:10.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com'
2022-02-04T17:46:10.531 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64
2022-02-04T17:46:10.587 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream"
2022-02-04T17:46:10.588 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8"
2022-02-04T17:46:10.588 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos"
2022-02-04T17:46:10.588 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora"
2022-02-04T17:46:10.588 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8"
2022-02-04T17:46:10.588 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8"
2022-02-04T17:46:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-04T17:46:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31"
2022-02-04T17:46:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-04T17:46:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/"
2022-02-04T17:46:10.590 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-04T17:46:10.590 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-04T17:46:10.590 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-04T17:46:10.591 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server
2022-02-04T17:46:10.609 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: set os type and version on server
2022-02-04T17:46:10.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2022-02-04T17:46:10.809 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64
2022-02-04T17:46:10.867 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="CentOS Stream"
2022-02-04T17:46:10.867 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8"
2022-02-04T17:46:10.867 INFO:teuthology.orchestra.run.smithi141.stdout:ID="centos"
2022-02-04T17:46:10.868 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="rhel fedora"
2022-02-04T17:46:10.868 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8"
2022-02-04T17:46:10.868 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8"
2022-02-04T17:46:10.868 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-04T17:46:10.869 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31"
2022-02-04T17:46:10.869 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-04T17:46:10.869 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://centos.org/"
2022-02-04T17:46:10.869 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-04T17:46:10.869 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-04T17:46:10.870 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-04T17:46:10.870 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server
2022-02-04T17:46:10.970 INFO:teuthology.dispatcher.supervisor:Running job 6661649
2022-02-04T17:46:10.971 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2022-02-04_17:39:45-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6661649 --name nojha-2022-02-04_17:39:45-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/nojha-2022-02-04_17:39:45-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6661649/orig.config.yaml
2022-02-04T17:46:10.974 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2022-02-04_17:39:45-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6661649
2022-02-04T17:46:10.975 INFO:teuthology.dispatcher.supervisor:Job PID: 21714
2022-02-04T17:46:10.975 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-05T00:16:21.976 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-05T00:16:22.475 INFO:teuthology.kill:Killing Pids: {21714}
2022-02-05T00:16:23.683 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2022-02-05T00:16:23.684 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2022-02-05T00:16:23.684 INFO:teuthology.misc:Compressing logs...
2022-02-05T00:16:23.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com'
2022-02-05T00:16:23.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com'
2022-02-05T00:16:24.170 INFO:teuthology.misc:Compressing logs...
2022-02-05T00:16:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-05T00:16:25.075 INFO:teuthology.misc:Compressing logs...
2022-02-05T00:16:30.800 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi141.qrcpzl.log: file size changed while zipping
2022-02-05T00:16:34.584 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-mon.smithi141.log: file size changed while zipping
2022-02-05T00:16:48.725 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-05T00:17:03.490 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-05T00:17:09.150 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi141.pmglvk.log: file size changed while zipping
2022-02-05T00:17:20.350 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-05T00:18:13.647 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi078.ypbjqh.log: file size changed while zipping
2022-02-05T00:18:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-05T00:18:38.305 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-05T00:18:40.661 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-mgr.smithi078.rwvpmm.log: file size changed while zipping
2022-02-05T00:18:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-mon.smithi078.log: file size changed while zipping
2022-02-05T00:18:57.951 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi078.mjajaq.log: file size changed while zipping
2022-02-05T00:19:11.306 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/22cc20d0-85e3-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-05T00:19:45.773 INFO:teuthology.orchestra.run.smithi078.stderr:tar: .: file changed as we read it
2022-02-05T00:19:57.295 INFO:teuthology.kill:No teuthology processes running
2022-02-05T00:19:58.514 INFO:teuthology.kill:Nuking machines: ['smithi078', 'smithi141']
2022-02-05T00:19:59.481 INFO:teuthology.kill:2022-02-05 00:19:59,480.480 INFO:teuthology.nuke:targets:
2022-02-05T00:19:59.482 INFO:teuthology.kill:  smithi078.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsdPxnW3JqhFbFTJVbdxdGLYDJpGEPHmmXUxVTlv0FBldzGlo55qVA8OwCCsBKBtIkIfeHC7+JN1kTqNULoI5lPRi45pXHK4EMa4rozg7FqTDkYNvWEq8vIgMV2aHJdVwMYkF6HK3Ax/+xVf6apBXsSNQYuZoxG+4l/VuO5oZ4KuMX66J7R2WAc8KB481Lws4ZI3D+I//B1prxJZMftxRu47046J1ZB6Y9gsQf/aFTPAO+hykURCIh00W1JMXcDrTPO9n6t4d+FzpcPvRI1E7Drf5GLAR1S2+ygZauUKsXStk67O7JP+Po8YFy1mkl84RNI++dkofliQmrDHvGl8V1KmgtTVVMmZx463jI8B4vp6io+wQLdECRR+En7oao+3PInE7yhxFSJh+7tg/DKEg1bcNtHzZu8Qi+Aim+y8ct0vwsLRjFAZovZ6bI7yVH3lGkf2nfRQB3A28WXLKAPMLl/QHqa/YyX4WCcfIJxrrB0nFDH8yTEh3IlHIDG+PM67E=
2022-02-05T00:19:59.482 INFO:teuthology.kill:  smithi141.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcSHaFKXpYCbjJi1IEBsSbnbE81Un3Rg4dZzEjYRU+/YbCwuvPnjtGRKbaYBKREiaGppJQD18mVT7PUZFBRrQkCPR6HCmuKKSLKgwTSfoymcx7ry/7BYUyfvtUkmNFkh6Oi8TGRIqDvD55P+huwC9HZ5ELJd0pvnO52lk/BsKn8+pxOk7pB1WyOxpn5vyRffITJrKKKLle4g5pZNR0GTkBD2gOvaExFxXaqXObPuUpEEeKMyl2nUTt0wGlCX4Pqre0e3LeK8zOj/SQ7JT0n1Fd1TsWdOs91of7yXsvwa+iTx1US76jitMSAeb1Z0DyjQw6lGgxjffbSViuFVL9RCiueTQ6xS4kIQo9c8wv9nT8qSkgCkG0HGPDjVDeXRleIReYCxXEvstF4BvVfdjHJp7TCBe4VKoEOLDBz3oTjJF63NWL+HJRJbZYbS2zUfdz4tZpiU3YSQ2ENh8cI1tiIqgOB6E7lf9njRRI6bdhRl0k8OC0cpmVWhnuZBW7suqWg1U=
2022-02-05T00:19:59.508 INFO:teuthology.kill:2022-02-05 00:19:59,508.508 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-05T00:19:59.510 INFO:teuthology.kill:2022-02-05 00:19:59,510.510 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-05T00:19:59.587 INFO:teuthology.kill:2022-02-05 00:19:59,587.587 INFO:teuthology.orchestra.console:Power off smithi141
2022-02-05T00:19:59.600 INFO:teuthology.kill:2022-02-05 00:19:59,600.600 INFO:teuthology.orchestra.console:Power off smithi078
2022-02-05T00:20:07.962 INFO:teuthology.kill:2022-02-05 00:20:07,961.961 INFO:teuthology.orchestra.console:Power off for smithi078 completed
2022-02-05T00:20:08.106 INFO:teuthology.kill:2022-02-05 00:20:08,106.106 INFO:teuthology.lock.ops:unlocked: smithi078.front.sepia.ceph.com
2022-02-05T00:20:12.064 INFO:teuthology.kill:2022-02-05 00:20:12,063.063 INFO:teuthology.orchestra.console:Power off for smithi141 completed
2022-02-05T00:20:12.209 INFO:teuthology.kill:2022-02-05 00:20:12,208.208 INFO:teuthology.lock.ops:unlocked: smithi141.front.sepia.ceph.com
2022-02-05T00:22:12.404 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-05T00:22:12.528 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi078 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-04_22:20:21-rados-wip-yuri8-testing-2022-02-04-1220-quincy-distro-default-smithi/6661973
2022-02-05T00:22:12.529 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi141 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-04_22:20:21-rados-wip-yuri8-testing-2022-02-04-1220-quincy-distro-default-smithi/6661973