2022-10-25T11:51:37.586 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-25T11:51:37.727 INFO:teuthology.lock.ops:Start node 'smithi018.front.sepia.ceph.com' reimaging 2022-10-25T11:51:37.729 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: reset os type and version on server 2022-10-25T11:51:37.729 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2022-10-25T11:51:37.750 INFO:teuthology.lock.ops:Node 'smithi018.front.sepia.ceph.com' reimaging is complete 2022-10-25T11:51:37.751 INFO:teuthology.lock.ops:Start node 'smithi142.front.sepia.ceph.com' reimaging 2022-10-25T11:51:37.751 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: reset os type and version on server 2022-10-25T11:51:37.752 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2022-10-25T11:51:37.779 INFO:teuthology.lock.ops:Node 'smithi142.front.sepia.ceph.com' reimaging is complete 2022-10-25T11:51:37.977 INFO:teuthology.provision.fog.smithi018:Scheduling deploy of centos 8.stream 2022-10-25T11:51:37.999 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of centos 8.stream 2022-10-25T11:51:38.383 INFO:teuthology.orchestra.console:Power off smithi018 2022-10-25T11:51:38.404 INFO:teuthology.orchestra.console:Power off smithi142 2022-10-25T11:51:38.536 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2022-10-25T11:51:38.637 INFO:teuthology.orchestra.console:Power on smithi018 2022-10-25T11:51:47.002 INFO:teuthology.orchestra.console:Power on for smithi018 completed 2022-10-25T11:51:47.103 INFO:teuthology.provision.fog.smithi018:Waiting for deploy to finish 2022-10-25T11:51:50.884 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2022-10-25T11:51:50.985 INFO:teuthology.orchestra.console:Power on smithi142 2022-10-25T11:51:55.238 INFO:teuthology.orchestra.console:Power on for smithi142 completed 2022-10-25T11:51:55.339 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish 2022-10-25T11:55:26.871 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-25T11:55:47.176 INFO:teuthology.provision.fog.smithi018:Node is ready 2022-10-25T11:55:47.204 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018.front.sepia.ceph.com 2022-10-25T11:55:47.267 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18 smithi018.front.sepia.ceph.com smithi018 2022-10-25T11:55:47.654 INFO:teuthology.provision.fog.smithi018:Deploy complete! 2022-10-25T11:56:39.834 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-25T11:56:40.113 INFO:teuthology.provision.fog.smithi142:Node is ready 2022-10-25T11:56:40.174 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142.front.sepia.ceph.com 2022-10-25T11:56:40.237 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142 smithi142.front.sepia.ceph.com smithi142 2022-10-25T11:56:40.605 INFO:teuthology.provision.fog.smithi142:Deploy complete! 2022-10-25T11:56:40.822 INFO:teuthology.lock.ops:Checking smithi018.front.sepia.ceph.com 2022-10-25T11:56:40.824 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-25T11:56:40.849 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com 2022-10-25T11:56:40.850 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-25T11:56:40.885 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: set os type and version on server 2022-10-25T11:56:40.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2022-10-25T11:56:41.289 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2022-10-25T11:56:41.351 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2022-10-25T11:56:41.351 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="8" 2022-10-25T11:56:41.352 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2022-10-25T11:56:41.352 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2022-10-25T11:56:41.352 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="8" 2022-10-25T11:56:41.352 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el8" 2022-10-25T11:56:41.353 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-25T11:56:41.353 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2022-10-25T11:56:41.353 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-25T11:56:41.353 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2022-10-25T11:56:41.354 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-25T11:56:41.354 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-25T11:56:41.355 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-25T11:56:41.357 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2022-10-25T11:56:41.394 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server 2022-10-25T11:56:41.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2022-10-25T11:56:41.740 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2022-10-25T11:56:41.802 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2022-10-25T11:56:41.803 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8" 2022-10-25T11:56:41.803 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2022-10-25T11:56:41.803 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2022-10-25T11:56:41.804 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8" 2022-10-25T11:56:41.804 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2022-10-25T11:56:41.804 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-25T11:56:41.805 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2022-10-25T11:56:41.805 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-25T11:56:41.806 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2022-10-25T11:56:41.806 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-25T11:56:41.807 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-25T11:56:41.807 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-25T11:56:41.808 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2022-10-25T11:56:41.972 INFO:teuthology.dispatcher.supervisor:Running job 7078351 2022-10-25T11:56:41.973 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-10-23_07:07:02-orch-quincy-distro-default-smithi/7078351 --name teuthology-2022-10-23_07:07:02-orch-quincy-distro-default-smithi --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/teuthology-2022-10-23_07:07:02-orch-quincy-distro-default-smithi/7078351/orig.config.yaml 2022-10-25T11:56:41.984 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-10-23_07:07:02-orch-quincy-distro-default-smithi/7078351 2022-10-25T11:56:41.985 INFO:teuthology.dispatcher.supervisor:Job PID: 29488 2022-10-25T11:56:41.985 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-10-25T23:57:06.435 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-10-25T23:57:06.638 INFO:teuthology.kill:Killing Pids: {29488} 2022-10-25T23:57:07.892 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2022-10-25T23:57:07.892 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2022-10-25T23:57:07.893 INFO:teuthology.misc:Compressing logs... 2022-10-25T23:57:07.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2022-10-25T23:57:08.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2022-10-25T23:57:08.389 INFO:teuthology.misc:Compressing logs... 2022-10-25T23:57:08.707 INFO:teuthology.misc:Compressing logs... 2022-10-25T23:57:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-10-25T23:57:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-mds.cephfs.smithi018.zinnqu.log: file size changed while zipping 2022-10-25T23:57:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-10-25T23:58:04.092 INFO:teuthology.orchestra.run.smithi018.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-10-25T23:58:05.382 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-mds.cephfs.smithi142.ssnnjj.log: file size changed while zipping 2022-10-25T23:58:25.743 INFO:teuthology.orchestra.run.smithi018.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-mon.smithi018.log: file size changed while zipping 2022-10-25T23:58:34.143 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-10-25T23:58:34.442 INFO:teuthology.orchestra.run.smithi018.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-mgr.smithi018.doiowj.log: file size changed while zipping 2022-10-25T23:58:40.725 INFO:teuthology.orchestra.run.smithi018.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-mds.cephfs.smithi018.gdgqqr.log: file size changed while zipping 2022-10-25T23:58:48.843 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-10-25T23:59:32.782 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-mds.cephfs.smithi142.crqjrl.log: file size changed while zipping 2022-10-25T23:59:56.851 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-10-26T00:00:04.458 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/b748aff0-545d-11ed-8438-001a4aab830c/ceph-mon.smithi142.log: file size changed while zipping 2022-10-26T00:00:58.024 INFO:teuthology.kill:No teuthology processes running 2022-10-26T00:00:59.346 INFO:teuthology.kill:Nuking machines: ['smithi018', 'smithi142'] 2022-10-26T00:01:00.435 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-10-26T00:01:00.436 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-10-26T00:01:00.436 INFO:teuthology.kill:2022-10-26 00:01:00,435.435 INFO:teuthology.nuke:targets: 2022-10-26T00:01:00.437 INFO:teuthology.kill: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdkO1rlf6IUiT/2wtZsBxE+kSeeVypu7+EvhsUvIngEO/laW7BlOD1cbNymvUW09O5jnYO+/lcYEYOE84pSJJfpnNuB9MFUTjaLVwwWmgM3OjO2+szWnfMm6kOgXrPnkRrhBa6C395AaRrprjB1e/6YUnFtV1dQ9mwEvdcoUDRxcE9T1l7oPxZe23bZzMIzVS3ZbU3ByR7M7F0GJmHOmH3E15rL7vlJfLLW1Mq/isJMoVRgcfxaltDTT20VHD4KTF8SM/GLvg+umgQ/4pINdF6ExGOQimWlZqj10qV3vSGwLgmXOUmY4ILJ31GEC2WjPojXqu9EJwsvgOPNBjYKaSv2dqg/Rye8eMVCdTFW4snDghZC5f5PHTmGbVyrmjt0Z9Py6dECFXygXCuEReQlJMCTnLJX9R3YmW7IFBjF7VzgmJPFxWBl2BfwwR8afftBesDYp0fZZL5xHGgtKsm49JJD7M80JCDvC1xwC09p8EqwUWfbXnEK+gCnWhBkXimmGE= 2022-10-26T00:01:00.437 INFO:teuthology.kill: smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfAaqRssS2pJjQmRRRghGINJ7t3PnQSmZJ8Ohl7HozCN46F6fGJ6/XoKk8gGQ25ezS9gyHgOtC2tBUI+UxIDvTXO9DPBSiFGYUvBLhklUcwbvyDeztuK4SAmUXBjTaoVKoLS9uDmAcYWV+6hyV+hJJgnLUDti9n6Xg6mDawixnBu2w25pRNBcU7Uknbj8WeDlEfec7VYt58PyzSlo7TuZ9lhFaOoLK9y9ew8KlLrPonz/fm0wjQhCPuE4nfg/q+85GuuJ7/3AQCRWIvHtA7KNCUSUGQ7DJj7vbxEPMxj3ysfanf6E8VJdDBiIuY93V81scVvL6y6PdpXnNP7LHEvtWBujL0aR9hbUWol+KUak+W6ThFS1yYx/WcKqNMJ8CxVODAdCtSXRY2hRV7dym2SxrOVxtSSutxc91ImeWZzEBD3juiNzTLK9msAS2k+nY514E1FUJnyxcQ2lKyes5la5XiGWm2fawN8rJM+GjPHbDX4lgd16aHjSRBEXYLES/Dos= 2022-10-26T00:01:00.437 INFO:teuthology.kill:2022-10-26 00:01:00,435.435 INFO:teuthology.nuke:Checking targets against current locks 2022-10-26T00:01:00.491 INFO:teuthology.kill:2022-10-26 00:01:00,490.490 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-26T00:01:00.513 INFO:teuthology.kill:2022-10-26 00:01:00,513.513 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-26T00:01:00.583 INFO:teuthology.kill:2022-10-26 00:01:00,583.583 INFO:teuthology.orchestra.console:Power off smithi018 2022-10-26T00:01:00.621 INFO:teuthology.kill:2022-10-26 00:01:00,621.621 INFO:teuthology.orchestra.console:Power off smithi142 2022-10-26T00:01:08.959 INFO:teuthology.kill:2022-10-26 00:01:08,959.959 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2022-10-26T00:01:09.115 INFO:teuthology.kill:2022-10-26 00:01:09,114.114 INFO:teuthology.lock.ops:unlocked: smithi018.front.sepia.ceph.com 2022-10-26T00:01:13.095 INFO:teuthology.kill:2022-10-26 00:01:13,095.095 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2022-10-26T00:01:13.243 INFO:teuthology.kill:2022-10-26 00:01:13,242.242 INFO:teuthology.lock.ops:unlocked: smithi142.front.sepia.ceph.com 2022-10-26T00:03:13.564 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15