2022-10-25T19:33:47.837 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9
2022-10-25T19:33:47.981 INFO:teuthology.lock.ops:Start node 'smithi043.front.sepia.ceph.com' reimaging
2022-10-25T19:33:47.982 INFO:teuthology.lock.ops:Updating [smithi043.front.sepia.ceph.com]: reset os type and version on server
2022-10-25T19:33:47.983 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server
2022-10-25T19:33:48.006 INFO:teuthology.lock.ops:Node 'smithi043.front.sepia.ceph.com' reimaging is complete
2022-10-25T19:33:48.007 INFO:teuthology.lock.ops:Start node 'smithi103.front.sepia.ceph.com' reimaging
2022-10-25T19:33:48.007 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: reset os type and version on server
2022-10-25T19:33:48.007 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server
2022-10-25T19:33:48.028 INFO:teuthology.lock.ops:Node 'smithi103.front.sepia.ceph.com' reimaging is complete
2022-10-25T19:33:48.229 INFO:teuthology.provision.fog.smithi043:Scheduling deploy of centos 8.stream
2022-10-25T19:33:48.243 INFO:teuthology.provision.fog.smithi103:Scheduling deploy of centos 8.stream
2022-10-25T19:33:48.547 INFO:teuthology.orchestra.console:Power off smithi043
2022-10-25T19:33:48.587 INFO:teuthology.orchestra.console:Power off smithi103
2022-10-25T19:33:56.927 INFO:teuthology.orchestra.console:Power off for smithi043 completed
2022-10-25T19:33:56.950 INFO:teuthology.orchestra.console:Power off for smithi103 completed
2022-10-25T19:33:57.030 INFO:teuthology.orchestra.console:Power on smithi043
2022-10-25T19:33:57.052 INFO:teuthology.orchestra.console:Power on smithi103
2022-10-25T19:34:05.409 INFO:teuthology.orchestra.console:Power on for smithi043 completed
2022-10-25T19:34:05.416 INFO:teuthology.orchestra.console:Power on for smithi103 completed
2022-10-25T19:34:05.512 INFO:teuthology.provision.fog.smithi043:Waiting for deploy to finish
2022-10-25T19:34:05.518 INFO:teuthology.provision.fog.smithi103:Waiting for deploy to finish
2022-10-25T19:38:10.336 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-10-25T19:38:10.603 INFO:teuthology.provision.fog.smithi043:Node is ready
2022-10-25T19:38:10.623 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043.front.sepia.ceph.com
2022-10-25T19:38:10.682 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43 smithi043.front.sepia.ceph.com smithi043
2022-10-25T19:38:11.031 INFO:teuthology.provision.fog.smithi043:Deploy complete!
2022-10-25T19:39:07.550 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-10-25T19:39:07.811 INFO:teuthology.provision.fog.smithi103:Node is ready
2022-10-25T19:39:07.870 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103.front.sepia.ceph.com
2022-10-25T19:39:07.928 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103 smithi103.front.sepia.ceph.com smithi103
2022-10-25T19:39:08.265 INFO:teuthology.provision.fog.smithi103:Deploy complete!
2022-10-25T19:39:08.459 INFO:teuthology.lock.ops:Checking smithi043.front.sepia.ceph.com
2022-10-25T19:39:08.460 INFO:teuthology.lock.ops:New key found. Updating...
2022-10-25T19:39:08.482 INFO:teuthology.lock.ops:Checking smithi103.front.sepia.ceph.com
2022-10-25T19:39:08.483 INFO:teuthology.lock.ops:New key found. Updating...
2022-10-25T19:39:08.507 INFO:teuthology.lock.ops:Updating [smithi043.front.sepia.ceph.com]: set os type and version on server
2022-10-25T19:39:08.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com'
2022-10-25T19:39:08.727 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64
2022-10-25T19:39:08.785 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream"
2022-10-25T19:39:08.785 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="8"
2022-10-25T19:39:08.786 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos"
2022-10-25T19:39:08.786 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora"
2022-10-25T19:39:08.786 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="8"
2022-10-25T19:39:08.786 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el8"
2022-10-25T19:39:08.787 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 8"
2022-10-25T19:39:08.787 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31"
2022-10-25T19:39:08.787 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-10-25T19:39:08.787 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/"
2022-10-25T19:39:08.788 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-10-25T19:39:08.788 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-10-25T19:39:08.788 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-10-25T19:39:08.789 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server
2022-10-25T19:39:08.809 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: set os type and version on server
2022-10-25T19:39:08.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com'
2022-10-25T19:39:09.020 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64
2022-10-25T19:39:09.077 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream"
2022-10-25T19:39:09.077 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="8"
2022-10-25T19:39:09.077 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos"
2022-10-25T19:39:09.078 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora"
2022-10-25T19:39:09.078 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="8"
2022-10-25T19:39:09.078 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el8"
2022-10-25T19:39:09.079 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 8"
2022-10-25T19:39:09.079 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31"
2022-10-25T19:39:09.079 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-10-25T19:39:09.079 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/"
2022-10-25T19:39:09.080 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-10-25T19:39:09.080 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-10-25T19:39:09.080 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-10-25T19:39:09.081 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server
2022-10-25T19:39:09.212 INFO:teuthology.dispatcher.supervisor:Running job 7080172
2022-10-25T19:39:09.213 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080172 --name teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080172/orig.config.yaml
2022-10-25T19:39:09.218 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080172
2022-10-25T19:39:09.218 INFO:teuthology.dispatcher.supervisor:Job PID: 30948
2022-10-25T19:39:09.219 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-10-26T07:39:58.873 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-10-26T07:39:59.695 INFO:teuthology.kill:Killing Pids: {30948}
2022-10-26T07:40:01.118 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2022-10-26T07:40:01.119 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2022-10-26T07:40:01.119 INFO:teuthology.misc:Compressing logs...
2022-10-26T07:40:01.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com'
2022-10-26T07:40:01.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com'
2022-10-26T07:40:01.623 INFO:teuthology.misc:Compressing logs...
2022-10-26T07:40:01.943 INFO:teuthology.misc:Compressing logs...
2022-10-26T07:40:04.733 INFO:teuthology.orchestra.run.smithi103.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-mon.smithi103.log: file size changed while zipping
2022-10-26T07:40:08.245 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-10-26T07:40:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-10-26T07:40:13.018 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-10-26T07:40:13.477 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-mds.cephfs.smithi043.nmuocg.log: file size changed while zipping
2022-10-26T07:40:15.285 INFO:teuthology.orchestra.run.smithi103.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-10-26T07:40:18.581 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-10-26T07:40:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-10-26T07:40:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-mgr.smithi043.sybxfc.log: file size changed while zipping
2022-10-26T07:40:22.437 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-mds.cephfs.smithi043.yfthii.log: file size changed while zipping
2022-10-26T07:40:30.590 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/073f6c34-549e-11ed-8438-001a4aab830c/ceph-mon.smithi043.log: file size changed while zipping
2022-10-26T07:41:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:tar: .: file changed as we read it
2022-10-26T07:41:24.932 INFO:teuthology.kill:No teuthology processes running
2022-10-26T07:41:26.210 INFO:teuthology.kill:Nuking machines: ['smithi043', 'smithi103']
2022-10-26T07:41:27.205 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-10-26T07:41:27.206 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-10-26T07:41:27.206 INFO:teuthology.kill:2022-10-26 07:41:27,204.204 INFO:teuthology.nuke:targets:
2022-10-26T07:41:27.206 INFO:teuthology.kill:  smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLTX0BdOzoPA7+yyydT81/GrH7wirBsAJY1RPVGe6uXXiKXKhZCY1OGsv+qoyrlGVEKlOE9pBRBXN/5m4LOCm28=
2022-10-26T07:41:27.207 INFO:teuthology.kill:  smithi103.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNZFOZ1Lku/ZA4yHkZw1bWYw5LKjdaF24m9/UmIMnkB7l/RvBxY8wg8uJvsuHDzoIBapAAuJLEs2XasfrIlqkP5BX27OL+R1gQludX4+B1kHodzHNCR/dwRGigw9mHAdYrFHMOPW64WqAplSV4VqAfjFKKra0uX/w3/hMOVoO3Yj+VBfraofHd7b5MC+5Vq3GtyYzPW1Tokd1y+6OU9m4ldDi2AZf6Lwz8sr7M7QDGeDxFu8U/uxL+rh/A5m0qPzZZ1zCurRur0t5V2G5Wol2lhwxfuYd/EUao/KbTOQyBetvnLIq3jNSFAOIjKJ1LMk2WpYjPPf01gUcmK+oXNvbmwtqCOjb5R3kjJohIEnzynQavgHGCshrk4OY41TxMdmTnWvNXENeGdQ/phKZbmFsW9CeV1G8z5+hu51eZPH51ShrXsq9K/09Ey6yEf8ggHfaa1S9pL5+M6xlaOnU3wWUxh4kiepJvuIROBheWKKn3wyA8LcXDoNtNV70uOv4BOVk=
2022-10-26T07:41:27.207 INFO:teuthology.kill:2022-10-26 07:41:27,204.204 INFO:teuthology.nuke:Checking targets against current locks
2022-10-26T07:41:27.257 INFO:teuthology.kill:2022-10-26 07:41:27,256.256 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-10-26T07:41:27.279 INFO:teuthology.kill:2022-10-26 07:41:27,279.279 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-10-26T07:41:27.347 INFO:teuthology.kill:2022-10-26 07:41:27,346.346 INFO:teuthology.orchestra.console:Power off smithi043
2022-10-26T07:41:27.385 INFO:teuthology.kill:2022-10-26 07:41:27,385.385 INFO:teuthology.orchestra.console:Power off smithi103
2022-10-26T07:41:35.718 INFO:teuthology.kill:2022-10-26 07:41:35,717.717 INFO:teuthology.orchestra.console:Power off for smithi043 completed
2022-10-26T07:41:35.741 INFO:teuthology.kill:2022-10-26 07:41:35,741.741 INFO:teuthology.orchestra.console:Power off for smithi103 completed
2022-10-26T07:41:35.887 INFO:teuthology.kill:2022-10-26 07:41:35,887.887 INFO:teuthology.lock.ops:unlocked: smithi043.front.sepia.ceph.com
2022-10-26T07:41:35.903 INFO:teuthology.kill:2022-10-26 07:41:35,903.903 INFO:teuthology.lock.ops:unlocked: smithi103.front.sepia.ceph.com
2022-10-26T07:43:36.152 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-10-26T07:43:36.264 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi103 but it was locked by another job: /home/teuthworker/archive/dis-2022-10-25_23:10:37-rbd-wip-dis-testing-distro-default-smithi/7080891