2021-12-28T08:28:19.902 INFO:root:teuthology version: 1.1.0-95a7d479
2021-12-28T08:28:20.028 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging
2021-12-28T08:28:20.029 INFO:teuthology.lock.ops:Updating [ubuntu@smithi002.front.sepia.ceph.com]: reset os type and version on server
2021-12-28T08:28:20.030 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2021-12-28T08:28:20.048 INFO:teuthology.lock.ops:Node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging is complete
2021-12-28T08:28:20.048 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging
2021-12-28T08:28:20.049 INFO:teuthology.lock.ops:Updating [ubuntu@smithi029.front.sepia.ceph.com]: reset os type and version on server
2021-12-28T08:28:20.049 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server
2021-12-28T08:28:20.067 INFO:teuthology.lock.ops:Node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging is complete
2021-12-28T08:28:20.258 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 8.stream
2021-12-28T08:28:20.293 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of centos 8.stream
2021-12-28T08:28:20.683 INFO:teuthology.orchestra.console:Power off smithi002
2021-12-28T08:28:20.741 INFO:teuthology.orchestra.console:Power off smithi029
2021-12-28T08:28:29.061 INFO:teuthology.orchestra.console:Power off for smithi002 completed
2021-12-28T08:28:29.108 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2021-12-28T08:28:29.163 INFO:teuthology.orchestra.console:Power on smithi002
2021-12-28T08:28:29.209 INFO:teuthology.orchestra.console:Power on smithi029
2021-12-28T08:28:37.529 INFO:teuthology.orchestra.console:Power on for smithi002 completed
2021-12-28T08:28:37.570 INFO:teuthology.orchestra.console:Power on for smithi029 completed
2021-12-28T08:28:37.631 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish
2021-12-28T08:28:37.671 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish
2021-12-28T08:31:24.195 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: Authentication failed.
2021-12-28T08:33:12.394 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-28T08:33:12.658 INFO:teuthology.provision.fog.smithi002:Node is ready
2021-12-28T08:33:12.675 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com
2021-12-28T08:33:12.732 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002
2021-12-28T08:33:13.068 INFO:teuthology.provision.fog.smithi002:Deploy complete!
2021-12-28T08:33:56.006 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-28T08:33:56.342 INFO:teuthology.provision.fog.smithi029:Node is ready
2021-12-28T08:33:56.361 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com
2021-12-28T08:33:56.419 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029
2021-12-28T08:33:56.823 INFO:teuthology.provision.fog.smithi029:Deploy complete!
2021-12-28T08:33:57.024 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com
2021-12-28T08:33:57.025 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-28T08:33:57.044 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com
2021-12-28T08:33:57.044 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-28T08:33:57.063 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server
2021-12-28T08:33:57.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com'
2021-12-28T08:33:57.311 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64
2021-12-28T08:33:57.369 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream"
2021-12-28T08:33:57.369 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8"
2021-12-28T08:33:57.370 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos"
2021-12-28T08:33:57.370 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora"
2021-12-28T08:33:57.370 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8"
2021-12-28T08:33:57.371 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8"
2021-12-28T08:33:57.371 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-28T08:33:57.371 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31"
2021-12-28T08:33:57.371 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-28T08:33:57.371 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/"
2021-12-28T08:33:57.372 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-28T08:33:57.372 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-28T08:33:57.372 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-28T08:33:57.373 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2021-12-28T08:33:57.392 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server
2021-12-28T08:33:57.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com'
2021-12-28T08:33:57.620 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64
2021-12-28T08:33:57.697 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream"
2021-12-28T08:33:57.697 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8"
2021-12-28T08:33:57.698 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos"
2021-12-28T08:33:57.698 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora"
2021-12-28T08:33:57.698 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8"
2021-12-28T08:33:57.698 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8"
2021-12-28T08:33:57.699 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-28T08:33:57.699 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31"
2021-12-28T08:33:57.699 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-28T08:33:57.699 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/"
2021-12-28T08:33:57.700 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-28T08:33:57.700 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-28T08:33:57.700 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-28T08:33:57.701 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server
2021-12-28T08:33:57.811 INFO:teuthology.dispatcher.supervisor:Running job 6586759
2021-12-28T08:33:57.811 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-12-28_03:15:03-fs-master-distro-default-smithi/6586759 --name teuthology-2021-12-28_03:15:03-fs-master-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/teuthology-2021-12-28_03:15:03-fs-master-distro-default-smithi/6586759/orig.config.yaml
2021-12-28T08:33:57.814 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-12-28_03:15:03-fs-master-distro-default-smithi/6586759
2021-12-28T08:33:57.815 INFO:teuthology.dispatcher.supervisor:Job PID: 12601
2021-12-28T08:33:57.816 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-28T20:34:17.517 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-28T20:34:17.703 INFO:teuthology.kill:Killing Pids: {12601}
2021-12-28T20:34:18.870 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-12-28T20:34:18.871 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-12-28T20:34:18.871 INFO:teuthology.misc:Compressing logs...
2021-12-28T20:34:18.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com'
2021-12-28T20:34:19.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com'
2021-12-28T20:34:19.382 INFO:teuthology.misc:Compressing logs...
2021-12-28T20:34:20.630 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-28T20:34:21.037 INFO:teuthology.misc:Compressing logs...
2021-12-28T20:34:29.156 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-mgr.smithi002.qjztfj.log: file size changed while zipping
2021-12-28T20:34:31.356 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-mds.cephfs.smithi029.plscow.log: file size changed while zipping
2021-12-28T20:34:49.702 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-28T20:35:17.300 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-28T20:35:33.887 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-28T20:35:34.012 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-mgr.smithi029.asntkp.log: file size changed while zipping
2021-12-28T20:35:43.057 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-mon.smithi029.log: file size changed while zipping
2021-12-28T20:36:17.342 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-mds.cephfs.smithi002.hobnfm.log: file size changed while zipping
2021-12-28T20:36:38.964 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-mon.smithi002.log: file size changed while zipping
2021-12-28T20:36:57.124 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-28T20:37:16.402 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-28T20:37:35.735 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/0ec13c8e-67ba-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-28T20:38:14.159 INFO:teuthology.orchestra.run.smithi002.stderr:tar: .: file changed as we read it
2021-12-28T20:38:29.748 INFO:teuthology.kill:No teuthology processes running
2021-12-28T20:38:30.935 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi029']
2021-12-28T20:38:31.998 INFO:teuthology.kill:2021-12-28 20:38:31,997.997 INFO:teuthology.nuke:targets:
2021-12-28T20:38:31.999 INFO:teuthology.kill:  smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkK/quxATp+kuSnbasDD93nt83+g522xFGfjG6o9+TqsvpFFH0rMvrKLcGJ/E9Tcs+/+B8ZrYevv/Yo4o7UQH7vMqXWKIJXz8kTMgcACGb2+WOZ7uAVKl2Y0SWwPvv42ASBXUsScDxCx7XglTQX6VmBHC/bGdHtCWPceJpmsUK6LyCDRgRsm+VOkRjiBTujWRL+8oSK4cRjA6XdJF4CheKiJ7GikVfAaD+fKGTpTrf3CSWKGrkCVKqXI8APjnxaK+mgRwVXZ9yYLSonzP2zrBSQzabg8M2BZiGe+G7xJoWMTQA+omuSf0fNxl0Aj7d0mYbiSWjXzAVOzKuQQ7PAMrLyXpHolCQYTKdC35gUH0gCZVI0WCtF29bU+QwE+cX9oiHzMTfnQ4xYVleTMvnJ7z8tcvzovcIFX++8drTAbMK9LnZKke4iPmgpRF804BRBLD2NYcr83aDWOAEbL80YPFvdRAADey6MkgXzNBlbkw7S1E8z0ZcELa86wanjogZSeU=
2021-12-28T20:38:31.999 INFO:teuthology.kill:  smithi029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIyu2Gu420HMbeVDLlyQeuX0/FAsuyFk2RjB16eeQN16n8hjV8GRKc2MyvrXVckqD2qGffcrBtj5hxTYdIMoqT+ues2r2TkCXwYmCA7RX7lAxMSPzfJv7rmmhVC1fHwvCgELjcTdFwGl7wn6xAl2locOhFzzKnvsBtXZF0ZNhXqEJHpKUIbT85PBenIeyr1AdLd7xd7Un/Lv0mSZGO0EIS9Wzy7rOdlltyqXafy3V/ZPH0XGLi4402Qixlp0pPtbREVgf9vYWpnwKvueaOUPqm2zfx+Is9pf9UpKBn66UK4MmD4RzXGWqdgZ8WRzkwS8mz5Q02IWIDqaak1l/JR4O2vfVjfCThPtKzG/SpbrwP61Wt0sD9AKHR1TNTNoG+A8qeafAeR1bEzcXFO40XCblGzLowfFlmEwsh9b90bZ5R10HS4LNsivoDPiBnQ3kXEnU1TN9sylBiYKKmju6395t0/L3cBOnd8ksLO8JPf5LwlP7xJ+vS3jZGWw1F3INbkX8=
2021-12-28T20:38:32.025 INFO:teuthology.kill:2021-12-28 20:38:32,024.024 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-28T20:38:32.028 INFO:teuthology.kill:2021-12-28 20:38:32,028.028 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-28T20:38:32.103 INFO:teuthology.kill:2021-12-28 20:38:32,103.103 INFO:teuthology.orchestra.console:Power off smithi029
2021-12-28T20:38:32.120 INFO:teuthology.kill:2021-12-28 20:38:32,119.119 INFO:teuthology.orchestra.console:Power off smithi002
2021-12-28T20:38:40.482 INFO:teuthology.kill:2021-12-28 20:38:40,481.481 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2021-12-28T20:38:40.628 INFO:teuthology.kill:2021-12-28 20:38:40,627.627 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com
2021-12-28T20:38:44.598 INFO:teuthology.kill:2021-12-28 20:38:44,597.597 INFO:teuthology.orchestra.console:Power off for smithi002 completed
2021-12-28T20:38:44.748 INFO:teuthology.kill:2021-12-28 20:38:44,748.748 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com
2021-12-28T20:40:44.969 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-28T20:40:45.084 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi002 but it was locked by another job: /home/teuthworker/archive/amathuri-2021-12-28_09:29:35-rados-master-distro-basic-smithi/6587594
2021-12-28T20:40:45.084 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi029 but it was locked by another job: /home/teuthworker/archive/amathuri-2021-12-28_09:29:35-rados-master-distro-basic-smithi/6587594