2022-07-26T19:10:24.176 INFO:root:teuthology version: 1.1.1.dev399+g5f965ce1 2022-07-26T19:10:24.321 INFO:teuthology.lock.ops:Start node 'smithi188.front.sepia.ceph.com' reimaging 2022-07-26T19:10:24.323 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: reset os type and version on server 2022-07-26T19:10:24.325 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2022-07-26T19:10:24.354 INFO:teuthology.lock.ops:Node 'smithi188.front.sepia.ceph.com' reimaging is complete 2022-07-26T19:10:24.354 INFO:teuthology.lock.ops:Start node 'smithi192.front.sepia.ceph.com' reimaging 2022-07-26T19:10:24.355 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: reset os type and version on server 2022-07-26T19:10:24.356 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2022-07-26T19:10:24.425 INFO:teuthology.lock.ops:Node 'smithi192.front.sepia.ceph.com' reimaging is complete 2022-07-26T19:10:24.613 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of centos 8.stream 2022-07-26T19:10:24.662 INFO:teuthology.provision.fog.smithi192:Scheduling deploy of centos 8.stream 2022-07-26T19:10:24.999 INFO:teuthology.orchestra.console:Power off smithi188 2022-07-26T19:10:25.069 INFO:teuthology.orchestra.console:Power off smithi192 2022-07-26T19:10:25.147 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2022-07-26T19:10:25.215 INFO:teuthology.orchestra.console:Power off for smithi192 completed 2022-07-26T19:10:25.249 INFO:teuthology.orchestra.console:Power on smithi188 2022-07-26T19:10:25.317 INFO:teuthology.orchestra.console:Power on smithi192 2022-07-26T19:10:29.577 INFO:teuthology.orchestra.console:Power on for smithi192 completed 2022-07-26T19:10:29.679 INFO:teuthology.provision.fog.smithi192:Waiting for deploy to finish 2022-07-26T19:10:33.618 INFO:teuthology.orchestra.console:Power on for smithi188 completed 2022-07-26T19:10:33.719 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish 2022-07-26T19:14:35.955 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-26T19:14:46.280 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-26T19:15:01.167 INFO:teuthology.provision.fog.smithi192:Node is ready 2022-07-26T19:15:01.194 INFO:teuthology.orchestra.run.smithi192.stdout:smithi192.front.sepia.ceph.com 2022-07-26T19:15:01.256 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.15.192 smithi192.front.sepia.ceph.com smithi192 2022-07-26T19:15:01.636 INFO:teuthology.provision.fog.smithi192:Deploy complete! 2022-07-26T19:15:12.085 INFO:teuthology.provision.fog.smithi188:Node is ready 2022-07-26T19:15:12.110 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188.front.sepia.ceph.com 2022-07-26T19:15:12.168 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188 2022-07-26T19:15:12.524 INFO:teuthology.provision.fog.smithi188:Deploy complete! 2022-07-26T19:15:12.705 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com 2022-07-26T19:15:12.706 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-26T19:15:12.733 INFO:teuthology.lock.ops:Checking smithi192.front.sepia.ceph.com 2022-07-26T19:15:12.734 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-26T19:15:12.754 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server 2022-07-26T19:15:12.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2022-07-26T19:15:12.971 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2022-07-26T19:15:13.030 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2022-07-26T19:15:13.030 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8" 2022-07-26T19:15:13.031 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2022-07-26T19:15:13.031 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2022-07-26T19:15:13.031 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8" 2022-07-26T19:15:13.032 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8" 2022-07-26T19:15:13.032 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-26T19:15:13.033 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2022-07-26T19:15:13.033 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-26T19:15:13.034 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2022-07-26T19:15:13.034 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-26T19:15:13.035 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-26T19:15:13.035 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-26T19:15:13.037 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2022-07-26T19:15:13.058 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: set os type and version on server 2022-07-26T19:15:13.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2022-07-26T19:15:13.367 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2022-07-26T19:15:13.425 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2022-07-26T19:15:13.425 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="8" 2022-07-26T19:15:13.425 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2022-07-26T19:15:13.426 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2022-07-26T19:15:13.426 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="8" 2022-07-26T19:15:13.426 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el8" 2022-07-26T19:15:13.426 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-26T19:15:13.427 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2022-07-26T19:15:13.427 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-26T19:15:13.427 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2022-07-26T19:15:13.428 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-26T19:15:13.428 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-26T19:15:13.428 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-26T19:15:13.429 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2022-07-26T19:15:13.596 INFO:teuthology.dispatcher.supervisor:Running job 6948417 2022-07-26T19:15:13.597 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_5f965ce157b8dcdaf2da92a5b5d1a63e787efdd9/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2022-07-26_18:31:55-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6948417 --name gabrioux-2022-07-26_18:31:55-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/gabrioux-2022-07-26_18:31:55-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6948417/orig.config.yaml 2022-07-26T19:15:13.602 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2022-07-26_18:31:55-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6948417 2022-07-26T19:15:13.603 INFO:teuthology.dispatcher.supervisor:Job PID: 22332 2022-07-26T19:15:13.603 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-27T07:16:10.023 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-27T07:16:10.207 INFO:teuthology.kill:Killing Pids: {22332} 2022-07-27T07:16:11.801 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2022-07-27T07:16:11.802 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2022-07-27T07:16:11.802 INFO:teuthology.misc:Compressing logs... 2022-07-27T07:16:11.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2022-07-27T07:16:12.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2022-07-27T07:16:12.342 INFO:teuthology.misc:Compressing logs... 2022-07-27T07:16:12.660 INFO:teuthology.misc:Compressing logs... 2022-07-27T07:16:13.537 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-mds.cephfs.smithi188.zacpgu.log: file size changed while zipping 2022-07-27T07:16:16.206 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-07-27T07:16:17.773 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-07-27T07:16:18.438 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-mon.smithi192.log: file size changed while zipping 2022-07-27T07:16:22.447 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-07-27T07:16:24.171 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-07-27T07:16:27.576 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-07-27T07:16:28.843 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-mon.smithi188.log: file size changed while zipping 2022-07-27T07:16:33.020 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-07-27T07:16:35.318 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/a3f89eb0-0d18-11ed-8430-001a4aab830c/ceph-mgr.smithi188.fdsnai.log: file size changed while zipping 2022-07-27T07:17:14.638 INFO:teuthology.orchestra.run.smithi188.stderr:tar: .: file changed as we read it 2022-07-27T07:17:28.201 INFO:teuthology.kill:No teuthology processes running 2022-07-27T07:17:29.846 INFO:teuthology.kill:Nuking machines: ['smithi188', 'smithi192'] 2022-07-27T07:17:30.963 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_5f965ce157b8dcdaf2da92a5b5d1a63e787efdd9/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-27T07:17:30.963 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-27T07:17:30.964 INFO:teuthology.kill:2022-07-27 07:17:30,962.962 INFO:teuthology.nuke:targets: 2022-07-27T07:17:30.964 INFO:teuthology.kill: smithi188.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDouAzXaufv26KHICLTx7d9mwn+g/r/KAyJhTDcCO9/s2bghupP/sCo0HqO6vkzioeUupDx4QGvkwqLHp4A8BGhJGm8uSHrZw6G/ZxCTlggvjU6w26fohpynsAt4gytON6nbh/fCvpJ9oUtu6IMgAARL6A059/StKwBhzJrfe2yO4gBrLYD0zaxHxyzmkL6w0tFEP69Pkci18Z9LeR6RfBrUX7/oHLyfJXC+Bh/ctAbgzfWzOl7xbNGD5hUcs0DSpE0Tu7pUnXVjWnWUz4VTzAXbmj9s+Jz8X4kjTdN8OJTj+nSByP8rK0YWcdT+aoKUBi58Umu8MHctGk7ycNkSmlG3cSmFfECc+B5DJ/LokBGwwh/5vsaH5v4qHzuCI4xJLJzmSLghYaEoHzN0FPubxHz8YWXqeLQjX6p0oKjBw4MAbDHnkWVHd6/73oLEzAka1NSaSDs55lTtizZZLt4AQIjNpaYvbRG7rwM2fXgO3YLosWzPXIcUJuE/EEkflfgvZE= 2022-07-27T07:17:30.964 INFO:teuthology.kill: smithi192.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv32VZo1ZVMvmrSo5l07TUuum0SIvWyHikrc87Uo0uoeFbyOsdoCqzOULZwTwFRe2Xzdzf6bGuytYAVbLVUcV2N/MXtuplnMvYhG/pIze0rbyX47SgHrfqQpBIBH9SB1ARqXDpbUU88lwDYdMC1/MdK7OAL+TzwjFAp8h4F5U3PlFotpkx6blEMFr2PWaXHOVAnmxFxH6mxSHBoXkXmex9y8mtRHgBRKiWOGASfAZSibLufv19HeWN2t8oByUvTsmceLAYL1FW04kI1ntrp+WzvD3jE0tn5q+ALoRY7GOwctVw8/cAaGXUdLmjrEHO2ejJGl3Fyof1LtCaNlc7hEZjHb7UFtQjdS1TzNXc4aq+vFwdPC2LZ5T1G+g0jBf4/xyx8grkOtc2QRjULWlcr9QtOawZCWtbDR1h9ZSx3b6NDhwvBGWz2nGTUaXJn2AvzjVUgQXL/PQH4Kt8TGVJyFH7riB6WorTxH70HUFW8D1UwGs5AqqrzfVeCxCpq2ktn0s= 2022-07-27T07:17:30.965 INFO:teuthology.kill:2022-07-27 07:17:30,962.962 INFO:teuthology.nuke:Checking targets against current locks 2022-07-27T07:17:31.016 INFO:teuthology.kill:2022-07-27 07:17:31,016.016 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-27T07:17:31.046 INFO:teuthology.kill:2022-07-27 07:17:31,046.046 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-27T07:17:31.115 INFO:teuthology.kill:2022-07-27 07:17:31,115.115 INFO:teuthology.orchestra.console:Power off smithi188 2022-07-27T07:17:31.156 INFO:teuthology.kill:2022-07-27 07:17:31,156.156 INFO:teuthology.orchestra.console:Power off smithi192 2022-07-27T07:17:39.492 INFO:teuthology.kill:2022-07-27 07:17:39,491.491 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2022-07-27T07:17:39.522 INFO:teuthology.kill:2022-07-27 07:17:39,522.522 INFO:teuthology.orchestra.console:Power off for smithi192 completed 2022-07-27T07:17:39.670 INFO:teuthology.kill:2022-07-27 07:17:39,669.669 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com 2022-07-27T07:17:39.675 INFO:teuthology.kill:2022-07-27 07:17:39,675.675 INFO:teuthology.lock.ops:unlocked: smithi192.front.sepia.ceph.com 2022-07-27T07:19:40.111 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-27T07:19:40.248 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi188 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-27_01:48:00-fs-wip-yuri6-testing-2022-07-26-1538-octopus-distro-default-smithi/6948872 2022-07-27T07:19:40.249 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi192 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-27_01:48:00-fs-wip-yuri6-testing-2022-07-26-1538-octopus-distro-default-smithi/6948872