2022-06-19T04:58:04.360 INFO:root:teuthology version: 1.1.1.dev374+g31a356e0 2022-06-19T04:58:04.523 INFO:teuthology.lock.ops:Start node 'smithi008.front.sepia.ceph.com' reimaging 2022-06-19T04:58:04.525 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: reset os type and version on server 2022-06-19T04:58:04.525 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2022-06-19T04:58:04.561 INFO:teuthology.lock.ops:Node 'smithi008.front.sepia.ceph.com' reimaging is complete 2022-06-19T04:58:04.561 INFO:teuthology.lock.ops:Start node 'smithi183.front.sepia.ceph.com' reimaging 2022-06-19T04:58:04.562 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: reset os type and version on server 2022-06-19T04:58:04.562 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2022-06-19T04:58:04.590 INFO:teuthology.lock.ops:Node 'smithi183.front.sepia.ceph.com' reimaging is complete 2022-06-19T04:58:04.787 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of centos 8.stream 2022-06-19T04:58:04.817 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of centos 8.stream 2022-06-19T04:58:05.142 INFO:teuthology.orchestra.console:Power off smithi008 2022-06-19T04:58:05.225 INFO:teuthology.orchestra.console:Power off smithi183 2022-06-19T04:58:13.524 INFO:teuthology.orchestra.console:Power off for smithi008 completed 2022-06-19T04:58:13.599 INFO:teuthology.orchestra.console:Power off for smithi183 completed 2022-06-19T04:58:13.625 INFO:teuthology.orchestra.console:Power on smithi008 2022-06-19T04:58:13.705 INFO:teuthology.orchestra.console:Power on smithi183 2022-06-19T04:58:22.012 INFO:teuthology.orchestra.console:Power on for smithi008 completed 2022-06-19T04:58:22.079 INFO:teuthology.orchestra.console:Power on for smithi183 completed 2022-06-19T04:58:22.114 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish 2022-06-19T04:58:22.180 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish 2022-06-19T05:02:20.339 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-19T05:02:40.339 INFO:teuthology.provision.fog.smithi008:Node is ready 2022-06-19T05:02:40.376 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com 2022-06-19T05:02:40.458 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008 2022-06-19T05:02:40.934 INFO:teuthology.provision.fog.smithi008:Deploy complete! 2022-06-19T05:02:55.051 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-19T05:02:55.380 INFO:teuthology.provision.fog.smithi183:Node is ready 2022-06-19T05:02:55.466 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com 2022-06-19T05:02:55.561 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183 2022-06-19T05:02:56.049 INFO:teuthology.provision.fog.smithi183:Deploy complete! 2022-06-19T05:02:56.546 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com 2022-06-19T05:02:56.547 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-19T05:02:56.589 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com 2022-06-19T05:02:56.590 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-19T05:02:56.624 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server 2022-06-19T05:02:56.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2022-06-19T05:02:56.980 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2022-06-19T05:02:57.052 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2022-06-19T05:02:57.053 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8" 2022-06-19T05:02:57.053 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2022-06-19T05:02:57.053 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2022-06-19T05:02:57.054 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8" 2022-06-19T05:02:57.054 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2022-06-19T05:02:57.054 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-19T05:02:57.054 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2022-06-19T05:02:57.055 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-19T05:02:57.059 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2022-06-19T05:02:57.060 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-19T05:02:57.060 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-19T05:02:57.060 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-19T05:02:57.062 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2022-06-19T05:02:57.101 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server 2022-06-19T05:02:57.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2022-06-19T05:02:57.498 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2022-06-19T05:02:57.570 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2022-06-19T05:02:57.571 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8" 2022-06-19T05:02:57.573 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2022-06-19T05:02:57.573 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2022-06-19T05:02:57.573 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8" 2022-06-19T05:02:57.574 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8" 2022-06-19T05:02:57.574 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-19T05:02:57.574 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2022-06-19T05:02:57.574 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-19T05:02:57.575 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2022-06-19T05:02:57.575 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-19T05:02:57.575 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-19T05:02:57.575 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-19T05:02:57.576 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2022-06-19T05:02:57.845 INFO:teuthology.dispatcher.supervisor:Running job 6885985 2022-06-19T05:02:57.846 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-06-19_03:31:03-rados-pacific-distro-default-smithi/6885985 --name teuthology-2022-06-19_03:31:03-rados-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/teuthology-2022-06-19_03:31:03-rados-pacific-distro-default-smithi/6885985/orig.config.yaml 2022-06-19T05:02:57.865 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-06-19_03:31:03-rados-pacific-distro-default-smithi/6885985 2022-06-19T05:02:57.866 INFO:teuthology.dispatcher.supervisor:Job PID: 31159 2022-06-19T05:02:57.867 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-06-19T17:04:05.270 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-06-19T17:04:05.446 INFO:teuthology.kill:Killing Pids: {31159} 2022-06-19T17:04:06.916 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2022-06-19T17:04:06.918 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2022-06-19T17:04:06.918 INFO:teuthology.misc:Compressing logs... 2022-06-19T17:04:06.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2022-06-19T17:04:07.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2022-06-19T17:04:07.450 INFO:teuthology.misc:Compressing logs... 2022-06-19T17:04:07.913 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-19T17:04:08.111 INFO:teuthology.misc:Compressing logs... 2022-06-19T17:04:11.024 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-mds.cephfs.smithi183.hwkynn.log: file size changed while zipping 2022-06-19T17:04:13.333 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-mgr.smithi008.pfcmus.log: file size changed while zipping 2022-06-19T17:04:27.332 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-06-19T17:04:29.013 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-06-19T17:04:51.035 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-06-19T17:04:52.807 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-mon.smithi008.log: file size changed while zipping 2022-06-19T17:04:58.132 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-mon.smithi183.log: file size changed while zipping 2022-06-19T17:05:00.541 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-mds.cephfs.smithi183.pbpgbj.log: file size changed while zipping 2022-06-19T17:05:19.945 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-06-19T17:05:21.277 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-mds.cephfs.smithi008.ppkisq.log: file size changed while zipping 2022-06-19T17:05:46.296 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-06-19T17:06:02.738 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/86586f2c-ef8e-11ec-8427-001a4aab830c/ceph-mds.cephfs.smithi008.mdniqh.log: file size changed while zipping 2022-06-19T17:06:41.307 INFO:teuthology.kill:No teuthology processes running 2022-06-19T17:06:42.628 INFO:teuthology.kill:Nuking machines: ['smithi008', 'smithi183'] 2022-06-19T17:06:43.645 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-06-19T17:06:43.646 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-06-19T17:06:43.647 INFO:teuthology.kill:2022-06-19 17:06:43,645.645 INFO:teuthology.nuke:targets: 2022-06-19T17:06:43.647 INFO:teuthology.kill: smithi008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCh1b9kOxh5GHmcgO/hHES1sZ6o2leCB9Mzf2CetJjZGRIsKWQKWo6IDanKmOxznrI+fXDlIQY4Ijl9fTR+/AqE60EAkT5A4Ky5IMRu7WmTvo6pSe3qiK5M6t/qDqFQAQOR47fzngcwfSLI5fTdkCT2sfdsWC8+YAaWw5jp/PeS9ZzSPg+jSOr4qwNLcto1NEezk24I9E5bTDsAGr5eoHgkj3rQLjhW4w0U+eAYUrPfRyQjAZAT1wbF7VLj3oLw0aplzT/j/qkzVg0hSZkuh9biYEJVY/WEdHKK+CcEe0tv9X8t29JaUsa/VDv2HxvZhSb/2krzZdm3GnMqf1qbO4CigGDwYMp9Emowmc42orBXQdwXQVm+6eIhqIVEdFVv1+P1fqirwy5vbMWNjCUqOEnu8ZNlQuD5pe+JodlGZUw6b2SEJrc0/wRHkwCMtmulynfnOE5hxkTwBQk07Iwb5//lRl+jyA/iQ8tDZsQ0uOBoT163zDp/crC9hb8D4m8Zs+E= 2022-06-19T17:06:43.647 INFO:teuthology.kill: smithi183.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiE/OMTQKV1x6i+Z5wL4B2xuIgq/as3yoTPzRxCrYEoXLXazUROWg+SqmxOObCgyElIYUYwo6vpMQW0I+24eOJbjSqdGDsWBOoOy3/Uj0uyPz6FY9dnUtYx5I63uznwGrsJZAXz7Ghc283BypsDU7m3zIqA195d2HwS+JDUJwC7dWjZCC6yY0u7321VU0lEHtZStVwCC128Y4Y9WPQQDQlebkByQ6823QOiN9W/0FFJjrHCehK3WzKaIuBTOwrehn3IrXNtU1gNLT27/FzI0CEBM3AZ47h5xDSFAgP6/shNGK2F9pasqnrzMm4PImXPvnRwm3wJ5R0EfhvRMDuF8vQrmWHvrzdrhL8UYSvEs3Mvhwgc3Ho3K+2gv3VFK/ETFJybpS5soy2Z0yh5L8SFE33CpX7SGDvQd84cLLAcle4L1uBmC8dgCxA/OpaXMRH9ryGhLvaOPRQRk3y/OCY0X+k3gsDwH7y53NXkFoP2ODEJ1mKzJZzQgn2CQ9eC+3Xkzs= 2022-06-19T17:06:43.648 INFO:teuthology.kill:2022-06-19 17:06:43,645.645 INFO:teuthology.nuke:Checking targets against current locks 2022-06-19T17:06:43.699 INFO:teuthology.kill:2022-06-19 17:06:43,698.698 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-19T17:06:43.735 INFO:teuthology.kill:2022-06-19 17:06:43,735.735 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-19T17:06:43.779 INFO:teuthology.kill:2022-06-19 17:06:43,779.779 INFO:teuthology.orchestra.console:Power off smithi008 2022-06-19T17:06:43.821 INFO:teuthology.kill:2022-06-19 17:06:43,821.821 INFO:teuthology.orchestra.console:Power off smithi183 2022-06-19T17:06:52.185 INFO:teuthology.kill:2022-06-19 17:06:52,184.184 INFO:teuthology.orchestra.console:Power off for smithi183 completed 2022-06-19T17:06:52.339 INFO:teuthology.kill:2022-06-19 17:06:52,338.338 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com 2022-06-19T17:06:56.293 INFO:teuthology.kill:2022-06-19 17:06:56,292.292 INFO:teuthology.orchestra.console:Power off for smithi008 completed 2022-06-19T17:06:56.443 INFO:teuthology.kill:2022-06-19 17:06:56,442.442 INFO:teuthology.lock.ops:unlocked: smithi008.front.sepia.ceph.com 2022-06-19T17:08:56.684 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15