2022-03-17T15:44:42.418 INFO:root:teuthology version: 1.1.0-a0900922
2022-03-17T15:44:42.555 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi101.front.sepia.ceph.com' reimaging
2022-03-17T15:44:42.556 INFO:teuthology.lock.ops:Updating [ubuntu@smithi101.front.sepia.ceph.com]: reset os type and version on server
2022-03-17T15:44:42.557 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2022-03-17T15:44:42.578 INFO:teuthology.lock.ops:Node 'ubuntu@smithi101.front.sepia.ceph.com' reimaging is complete
2022-03-17T15:44:42.578 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi159.front.sepia.ceph.com' reimaging
2022-03-17T15:44:42.579 INFO:teuthology.lock.ops:Updating [ubuntu@smithi159.front.sepia.ceph.com]: reset os type and version on server
2022-03-17T15:44:42.579 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2022-03-17T15:44:42.606 INFO:teuthology.lock.ops:Node 'ubuntu@smithi159.front.sepia.ceph.com' reimaging is complete
2022-03-17T15:44:42.793 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of centos 8.stream
2022-03-17T15:44:42.818 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 8.stream
2022-03-17T15:44:43.159 INFO:teuthology.orchestra.console:Power off smithi159
2022-03-17T15:44:43.176 INFO:teuthology.orchestra.console:Power off smithi101
2022-03-17T15:44:55.640 INFO:teuthology.orchestra.console:Power off for smithi159 completed
2022-03-17T15:44:55.651 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2022-03-17T15:44:55.742 INFO:teuthology.orchestra.console:Power on smithi159
2022-03-17T15:44:55.752 INFO:teuthology.orchestra.console:Power on smithi101
2022-03-17T15:45:04.110 INFO:teuthology.orchestra.console:Power on for smithi159 completed
2022-03-17T15:45:04.117 INFO:teuthology.orchestra.console:Power on for smithi101 completed
2022-03-17T15:45:04.212 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish
2022-03-17T15:45:04.218 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish
2022-03-17T15:47:35.828 ERROR:teuthology.orchestra.connection:Error authenticating with smithi159.front.sepia.ceph.com: Authentication failed.
2022-03-17T15:49:11.819 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-17T15:49:36.760 INFO:teuthology.provision.fog.smithi101:Node is ready
2022-03-17T15:49:36.783 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com
2022-03-17T15:49:36.843 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101
2022-03-17T15:49:37.216 INFO:teuthology.provision.fog.smithi101:Deploy complete!
2022-03-17T15:49:49.160 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-17T15:49:49.402 INFO:teuthology.provision.fog.smithi159:Node is ready
2022-03-17T15:49:49.459 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com
2022-03-17T15:49:49.517 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159
2022-03-17T15:49:49.857 INFO:teuthology.provision.fog.smithi159:Deploy complete!
2022-03-17T15:49:50.008 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com
2022-03-17T15:49:50.009 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-17T15:49:50.028 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com
2022-03-17T15:49:50.028 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-17T15:49:50.047 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server
2022-03-17T15:49:50.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2022-03-17T15:49:50.295 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64
2022-03-17T15:49:50.353 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="CentOS Stream"
2022-03-17T15:49:50.353 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8"
2022-03-17T15:49:50.353 INFO:teuthology.orchestra.run.smithi101.stdout:ID="centos"
2022-03-17T15:49:50.354 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="rhel fedora"
2022-03-17T15:49:50.354 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8"
2022-03-17T15:49:50.354 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8"
2022-03-17T15:49:50.354 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-17T15:49:50.354 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31"
2022-03-17T15:49:50.355 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-17T15:49:50.355 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://centos.org/"
2022-03-17T15:49:50.355 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-17T15:49:50.355 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-17T15:49:50.356 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-17T15:49:50.357 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2022-03-17T15:49:50.375 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server
2022-03-17T15:49:50.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com'
2022-03-17T15:49:50.568 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64
2022-03-17T15:49:50.626 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream"
2022-03-17T15:49:50.627 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8"
2022-03-17T15:49:50.627 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos"
2022-03-17T15:49:50.627 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora"
2022-03-17T15:49:50.627 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8"
2022-03-17T15:49:50.628 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8"
2022-03-17T15:49:50.628 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-17T15:49:50.628 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2022-03-17T15:49:50.628 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-17T15:49:50.629 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/"
2022-03-17T15:49:50.629 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-17T15:49:50.629 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-17T15:49:50.629 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-17T15:49:50.630 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2022-03-17T15:49:50.742 INFO:teuthology.dispatcher.supervisor:Running job 6742239
2022-03-17T15:49:50.742 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a09009223dc44ca923b2e6ad4cd4c4fe88381b22/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-17_14:54:32-rados-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/6742239 --name yuriw-2022-03-17_14:54:32-rados-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2022-03-17_14:54:32-rados-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/6742239/orig.config.yaml
2022-03-17T15:49:50.746 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-17_14:54:32-rados-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/6742239
2022-03-17T15:49:50.747 INFO:teuthology.dispatcher.supervisor:Job PID: 31941
2022-03-17T15:49:50.748 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-17T22:20:02.158 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-17T22:20:02.343 INFO:teuthology.kill:Killing Pids: {31941}
2022-03-17T22:20:03.609 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2022-03-17T22:20:03.610 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2022-03-17T22:20:03.611 INFO:teuthology.misc:Compressing logs...
2022-03-17T22:20:03.611 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2022-03-17T22:20:03.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com'
2022-03-17T22:20:04.113 INFO:teuthology.misc:Compressing logs...
2022-03-17T22:20:04.415 INFO:teuthology.misc:Compressing logs...
2022-03-17T22:20:06.643 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-mgr.smithi159.amyldd.log: file size changed while zipping
2022-03-17T22:20:20.446 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-03-17T22:20:31.369 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-03-17T22:21:30.461 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-mon.smithi101.log: file size changed while zipping
2022-03-17T22:21:39.839 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi159.vhegsf.log: file size changed while zipping
2022-03-17T22:21:57.201 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi159.mjgxxa.log: file size changed while zipping
2022-03-17T22:22:00.106 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-03-17T22:22:10.259 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-03-17T22:22:25.729 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-03-17T22:22:29.800 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-mon.smithi159.log: file size changed while zipping
2022-03-17T22:22:39.067 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/718a6c20-a60b-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-03-17T22:23:01.860 INFO:teuthology.orchestra.run.smithi101.stderr:tar: .: file changed as we read it
2022-03-17T22:23:32.863 INFO:teuthology.kill:No teuthology processes running
2022-03-17T22:23:34.036 INFO:teuthology.kill:Nuking machines: ['smithi101', 'smithi159']
2022-03-17T22:23:34.994 INFO:teuthology.kill:2022-03-17 22:23:34,994.994 INFO:teuthology.nuke:targets:
2022-03-17T22:23:34.995 INFO:teuthology.kill:  smithi101.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbXoSoufRejth6OyKYelTQ/zD0xCb9Y2KwVC+OfnOovswGIQlkr7YKyL1uBm/tnzhcKgfgSeEtiSR+sN4+L83RvXoPm0Hx9PCLfPsXN6dPfhz4k5WYGgYf1Pboa8UfnzhPDgJ1vurGQqeI/4oyEzsNqJ8vvEhjtFVjJR8bxhglkXnsv/FyJ2UBu0GUTI2VY2QegW4XCWCwwcYPP073hG2FT78gHVOksZEYrvsiv8Yl+IxXKI9vxNrk7TmANaKe1A17WkcDTgbRWCmf73RwTX6PVGqCrmSmffrelzi/cC4lN4WjowknD5AzGmWM1mqSSlwciam08nrQGu9pcs/Se4aH/jvJlyVl3DJP1C14BKKnAQBuE6+UdzKkEI8v0SzCUHYB1pPTiiKENOto4lLpwhN964sVBmRXg6HofRqizcL1w4ZOunMfO5l9ViL4YK79ps3si6cSVCiFJBDHYOYblR/3tXr37IyjacAXA5MvZWM79LjL4QN/8neEcujMKh0pHB8=
2022-03-17T22:23:34.996 INFO:teuthology.kill:  smithi159.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0Zpdml7fwSznjfzmF9IVTiFZDEnslzKUuiJCoPm9ZAkDbTfOK7XutTEfHgaHfIBJlQIXbOCdeaqNGgV6xrRvTmMSB+vE6F/KSZbiH6wV7NfgpXfCVAPOnfJLz5tiMrtlKX0DVYtxl7tAlydlE3cqgjyozLhB7ITIeMK9rTZ3OklR1iVib+WRLQTv67yAujVWLuRg2ccUdRsJMyMkTLsKMRI765LvfNYZhEnb/0b/bGzlpMTJ4erwX7uiJgzbdfhp/+s1VFnKziG7J4ekm4A2KncbJwzDGuXpzy6k2oFFTGlUSUH2Obg4/SlobfKeNAoAL43q87SuuaCcPWGIzdYUPZkKjsNm+GUUtCz4+qm6m1xe7HMhvTFzQ04u0aW6hIpIT7LxKTcHqexRSvBQ3R52HtIQ1wExVVSrn/maoDw+FLr3rYn8N/SuaZmFXyRFxpzAFR2ho/kyl0mq7ixHKQtnqAlhBGlyt7Py+OwLNigDjYKo6l8H/ClD0sdf7jyzV2a0=
2022-03-17T22:23:35.022 INFO:teuthology.kill:2022-03-17 22:23:35,022.022 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-17T22:23:35.026 INFO:teuthology.kill:2022-03-17 22:23:35,025.025 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-17T22:23:35.103 INFO:teuthology.kill:2022-03-17 22:23:35,103.103 INFO:teuthology.orchestra.console:Power off smithi101
2022-03-17T22:23:35.117 INFO:teuthology.kill:2022-03-17 22:23:35,116.116 INFO:teuthology.orchestra.console:Power off smithi159
2022-03-17T22:23:47.580 INFO:teuthology.kill:2022-03-17 22:23:47,579.579 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2022-03-17T22:23:47.585 INFO:teuthology.kill:2022-03-17 22:23:47,585.585 INFO:teuthology.orchestra.console:Power off for smithi159 completed
2022-03-17T22:23:47.729 INFO:teuthology.kill:2022-03-17 22:23:47,728.728 INFO:teuthology.lock.ops:unlocked: smithi101.front.sepia.ceph.com
2022-03-17T22:23:47.732 INFO:teuthology.kill:2022-03-17 22:23:47,732.732 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com
2022-03-17T22:25:47.990 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-17T22:25:48.109 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi101 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-03-17_17:32:48-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6742833
2022-03-17T22:25:48.109 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi159 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-03-17_17:32:48-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6742833