2022-02-01T22:09:53.571 INFO:root:teuthology version: 1.1.0-0f634dc1
2022-02-01T22:09:53.700 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi089.front.sepia.ceph.com' reimaging
2022-02-01T22:09:53.701 INFO:teuthology.lock.ops:Updating [ubuntu@smithi089.front.sepia.ceph.com]: reset os type and version on server
2022-02-01T22:09:53.702 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2022-02-01T22:09:53.722 INFO:teuthology.lock.ops:Node 'ubuntu@smithi089.front.sepia.ceph.com' reimaging is complete
2022-02-01T22:09:53.722 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging
2022-02-01T22:09:53.723 INFO:teuthology.lock.ops:Updating [ubuntu@smithi106.front.sepia.ceph.com]: reset os type and version on server
2022-02-01T22:09:53.723 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2022-02-01T22:09:53.742 INFO:teuthology.lock.ops:Node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging is complete
2022-02-01T22:09:53.929 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of centos 8.stream
2022-02-01T22:09:53.967 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of centos 8.stream
2022-02-01T22:09:54.339 INFO:teuthology.orchestra.console:Power off smithi089
2022-02-01T22:09:54.376 INFO:teuthology.orchestra.console:Power off smithi106
2022-02-01T22:10:06.827 INFO:teuthology.orchestra.console:Power off for smithi089 completed
2022-02-01T22:10:06.847 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2022-02-01T22:10:06.929 INFO:teuthology.orchestra.console:Power on smithi089
2022-02-01T22:10:06.954 INFO:teuthology.orchestra.console:Power on smithi106
2022-02-01T22:10:11.182 INFO:teuthology.orchestra.console:Power on for smithi089 completed
2022-02-01T22:10:11.283 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish
2022-02-01T22:10:15.319 INFO:teuthology.orchestra.console:Power on for smithi106 completed
2022-02-01T22:10:15.421 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish
2022-02-01T22:14:32.780 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-01T22:14:56.373 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-01T22:14:56.648 INFO:teuthology.provision.fog.smithi106:Node is ready
2022-02-01T22:14:56.665 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com
2022-02-01T22:14:56.723 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106
2022-02-01T22:14:57.058 INFO:teuthology.provision.fog.smithi106:Deploy complete!
2022-02-01T22:14:58.043 INFO:teuthology.provision.fog.smithi089:Node is ready
2022-02-01T22:14:58.065 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com
2022-02-01T22:14:58.123 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089
2022-02-01T22:14:58.472 INFO:teuthology.provision.fog.smithi089:Deploy complete!
2022-02-01T22:14:58.658 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com
2022-02-01T22:14:58.659 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-01T22:14:58.678 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com
2022-02-01T22:14:58.679 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-01T22:14:58.699 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server
2022-02-01T22:14:58.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com'
2022-02-01T22:14:58.927 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64
2022-02-01T22:14:58.982 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream"
2022-02-01T22:14:58.982 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8"
2022-02-01T22:14:58.983 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos"
2022-02-01T22:14:58.983 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora"
2022-02-01T22:14:58.983 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8"
2022-02-01T22:14:58.983 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8"
2022-02-01T22:14:58.984 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-01T22:14:58.984 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31"
2022-02-01T22:14:58.984 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-01T22:14:58.984 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/"
2022-02-01T22:14:58.985 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-01T22:14:58.985 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-01T22:14:58.985 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-01T22:14:58.986 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2022-02-01T22:14:59.005 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server
2022-02-01T22:14:59.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com'
2022-02-01T22:14:59.204 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64
2022-02-01T22:14:59.262 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream"
2022-02-01T22:14:59.263 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8"
2022-02-01T22:14:59.263 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos"
2022-02-01T22:14:59.263 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora"
2022-02-01T22:14:59.264 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8"
2022-02-01T22:14:59.264 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8"
2022-02-01T22:14:59.264 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-01T22:14:59.264 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31"
2022-02-01T22:14:59.264 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-01T22:14:59.265 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/"
2022-02-01T22:14:59.265 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-01T22:14:59.265 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-01T22:14:59.265 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-01T22:14:59.266 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2022-02-01T22:14:59.379 INFO:teuthology.dispatcher.supervisor:Running job 6653370
2022-02-01T22:14:59.380 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2022-02-01_20:31:43-rados-wip-mgr-fixes-pacific-distro-basic-smithi/6653370 --name nojha-2022-02-01_20:31:43-rados-wip-mgr-fixes-pacific-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/nojha-2022-02-01_20:31:43-rados-wip-mgr-fixes-pacific-distro-basic-smithi/6653370/orig.config.yaml
2022-02-01T22:14:59.384 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2022-02-01_20:31:43-rados-wip-mgr-fixes-pacific-distro-basic-smithi/6653370
2022-02-01T22:14:59.385 INFO:teuthology.dispatcher.supervisor:Job PID: 19359
2022-02-01T22:14:59.385 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-02T04:45:28.939 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-02T04:45:29.164 INFO:teuthology.kill:Killing Pids: {19359}
2022-02-02T04:45:30.434 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2022-02-02T04:45:30.435 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2022-02-02T04:45:30.435 INFO:teuthology.misc:Compressing logs...
2022-02-02T04:45:30.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com'
2022-02-02T04:45:30.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com'
2022-02-02T04:45:30.932 INFO:teuthology.misc:Compressing logs...
2022-02-02T04:45:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-02T04:45:31.991 INFO:teuthology.misc:Compressing logs...
2022-02-02T04:45:33.396 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-02T04:45:34.360 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-mon.smithi089.log: file size changed while zipping
2022-02-02T04:45:35.210 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-02T04:45:35.314 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-mgr.smithi089.prpuxe.log: file size changed while zipping
2022-02-02T04:45:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-mon.smithi106.log: file size changed while zipping
2022-02-02T04:45:36.825 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-02T04:45:37.273 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-02T04:45:38.265 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-02T04:45:40.088 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/dd2ba0e6-83ad-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-02T04:46:22.105 INFO:teuthology.orchestra.run.smithi106.stderr:tar: ./dd2ba0e6-83ad-11ec-8c35-001a4aab830c: file changed as we read it
2022-02-02T04:46:22.129 INFO:teuthology.orchestra.run.smithi106.stderr:tar: .: file changed as we read it
2022-02-02T04:46:22.207 INFO:teuthology.kill:No teuthology processes running
2022-02-02T04:46:23.333 INFO:teuthology.kill:Nuking machines: ['smithi089', 'smithi106']
2022-02-02T04:46:24.343 INFO:teuthology.kill:2022-02-02 04:46:24,343.343 INFO:teuthology.nuke:targets:
2022-02-02T04:46:24.346 INFO:teuthology.kill:  smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeZRus90DQ98Ty0Fti+e8FfcOj4aghqUmwRVhXvKtFH8bG1DP5BMxyxleYeAZSbRGKzDU92npnEaooRjPd7uwkCfm4uIR7dLUv1ycG2kGaqRgW0rT6Ehu46Tej3KDHjEeCEoMtxSlPMJowPAeVdYxM2Ss68AZNhpn1X4cWYnUO9S74qTXZy8q9P3ylVyVWs0USG9tTsEI6fVipMyD8aS1xZMQ70uObiHfQg1exvEf2J0iW0wpEoKgZ+Az+XhoTlAWHoXtUTTnCzhiHQumMdklpTz9PQavYtk+sw+j7dE/DoNGlMSBSJwXOX0qFA2nakHmk8w4O4sf4pohaj1YLn4GACdU0pQZ5xm6UHK/AtDlXIFaxcLSCoh0F/WKimSdo2Enj7nTfbq7KACWMw4SxszCiXHcH3rrbzbt9hObgDoTc7oUs+x1/ZP2UsNGjdJ2sWDWwzIP30Vvp8gzTRL9rrrx8yTy9KnJF9Daxp0ko1/Zacgkt0N7lCC7M8z5fthEcSX0=
2022-02-02T04:46:24.346 INFO:teuthology.kill:  smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsioobNWlcsoDyyyrfedfT5evgY/pzSskuJhqW6aos9HmLHA8tMwWiayPHt/6nOtfD36IyXebx94aFJqkqSItnbjoi3oRgxVKor6Q/kU07clAAb9tlcrdQ0vKmXE69+asHWxkOh0jAX8II2mPe0ZQqYgT6ibTGyMt4bdUtDJeUpiCP0ZrftlLUmgCYJ4TcG+jpEMNObBa0PzhxZKvN3M9lrMErZXohfMlitY3Q/8s3ana5dWCPArcYS/YR+45EQhuVIHTV7SQFPJbNqjabOinoPht/dEdOBbJSgCjC9ylc1gZ2Gwqb3uNS/a0GqnXMyZKeVuvOPbWNB6k9yHm5/bVP9Hs8QRK0pS2DHHIWblEuPWWncAp+pEbVlgN1vg+mps5+A1W++r0IGP722vaAD822juK1KR3tCnvP21FWS11jkXtyGyL9tn2K1Th4UzTql9EgRZwZxJp+7EPWcoh3gz2029jQtVL+9lDLnyRH8aX1oWROE5nJAkua77gwdYEmJ9s=
2022-02-02T04:46:24.370 INFO:teuthology.kill:2022-02-02 04:46:24,370.370 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-02T04:46:24.373 INFO:teuthology.kill:2022-02-02 04:46:24,373.373 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-02T04:46:24.449 INFO:teuthology.kill:2022-02-02 04:46:24,448.448 INFO:teuthology.orchestra.console:Power off smithi089
2022-02-02T04:46:24.462 INFO:teuthology.kill:2022-02-02 04:46:24,461.461 INFO:teuthology.orchestra.console:Power off smithi106
2022-02-02T04:46:36.932 INFO:teuthology.kill:2022-02-02 04:46:36,931.931 INFO:teuthology.orchestra.console:Power off for smithi089 completed
2022-02-02T04:46:36.937 INFO:teuthology.kill:2022-02-02 04:46:36,937.937 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2022-02-02T04:46:37.081 INFO:teuthology.kill:2022-02-02 04:46:37,080.080 INFO:teuthology.lock.ops:unlocked: smithi106.front.sepia.ceph.com
2022-02-02T04:46:37.084 INFO:teuthology.kill:2022-02-02 04:46:37,084.084 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com
2022-02-02T04:48:37.419 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-02T04:48:37.541 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi089 but it was locked by another job: /home/teuthworker/archive/dis-2022-02-02_01:08:13-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6654245
2022-02-02T04:48:37.542 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi106 but it was locked by another job: /home/teuthworker/archive/dis-2022-02-02_01:08:13-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6654245