2022-07-06T13:03:51.018 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-07-06T13:03:51.214 INFO:teuthology.lock.ops:Start node 'smithi055.front.sepia.ceph.com' reimaging 2022-07-06T13:03:51.216 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: reset os type and version on server 2022-07-06T13:03:51.216 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2022-07-06T13:03:51.253 INFO:teuthology.lock.ops:Node 'smithi055.front.sepia.ceph.com' reimaging is complete 2022-07-06T13:03:51.253 INFO:teuthology.lock.ops:Start node 'smithi075.front.sepia.ceph.com' reimaging 2022-07-06T13:03:51.254 INFO:teuthology.lock.ops:Updating [smithi075.front.sepia.ceph.com]: reset os type and version on server 2022-07-06T13:03:51.254 INFO:teuthology.lock.ops:Updating smithi075.front.sepia.ceph.com on lock server 2022-07-06T13:03:51.278 INFO:teuthology.lock.ops:Node 'smithi075.front.sepia.ceph.com' reimaging is complete 2022-07-06T13:03:51.502 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of centos 8.stream 2022-07-06T13:03:51.563 INFO:teuthology.provision.fog.smithi075:Scheduling deploy of centos 8.stream 2022-07-06T13:03:51.925 INFO:teuthology.orchestra.console:Power off smithi055 2022-07-06T13:03:51.971 INFO:teuthology.orchestra.console:Power off smithi075 2022-07-06T13:03:52.085 INFO:teuthology.orchestra.console:Power off for smithi055 completed 2022-07-06T13:03:52.119 INFO:teuthology.orchestra.console:Power off for smithi075 completed 2022-07-06T13:03:52.191 INFO:teuthology.orchestra.console:Power on smithi055 2022-07-06T13:03:52.236 INFO:teuthology.orchestra.console:Power on smithi075 2022-07-06T13:03:56.507 INFO:teuthology.orchestra.console:Power on for smithi075 completed 2022-07-06T13:03:56.609 INFO:teuthology.provision.fog.smithi075:Waiting for deploy to finish 2022-07-06T13:04:00.580 INFO:teuthology.orchestra.console:Power on for smithi055 completed 2022-07-06T13:04:00.681 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish 2022-07-06T13:07:43.347 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-06T13:07:59.307 INFO:teuthology.provision.fog.smithi055:Node is ready 2022-07-06T13:07:59.355 INFO:teuthology.orchestra.run.smithi055.stdout:smithi115 2022-07-06T13:07:59.425 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.115 smithi115.front.sepia.ceph.com smithi115 2022-07-06T13:07:59.856 INFO:teuthology.provision.fog.smithi055:Deploy complete! 2022-07-06T13:08:28.887 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-06T13:08:29.159 INFO:teuthology.provision.fog.smithi075:Node is ready 2022-07-06T13:08:29.181 INFO:teuthology.orchestra.run.smithi075.stdout:smithi075.front.sepia.ceph.com 2022-07-06T13:08:29.242 INFO:teuthology.orchestra.run.smithi075.stdout:172.21.15.75 smithi075.front.sepia.ceph.com smithi075 2022-07-06T13:08:29.616 INFO:teuthology.provision.fog.smithi075:Deploy complete! 2022-07-06T13:08:29.853 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com 2022-07-06T13:08:29.857 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-06T13:08:29.879 INFO:teuthology.lock.ops:Checking smithi075.front.sepia.ceph.com 2022-07-06T13:08:29.880 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-06T13:08:29.907 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server 2022-07-06T13:08:29.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2022-07-06T13:08:30.250 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2022-07-06T13:08:30.311 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2022-07-06T13:08:30.312 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8" 2022-07-06T13:08:30.322 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2022-07-06T13:08:30.323 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2022-07-06T13:08:30.323 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8" 2022-07-06T13:08:30.324 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8" 2022-07-06T13:08:30.324 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-06T13:08:30.325 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2022-07-06T13:08:30.325 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-06T13:08:30.325 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2022-07-06T13:08:30.326 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-06T13:08:30.327 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-06T13:08:30.327 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-06T13:08:30.330 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2022-07-06T13:08:30.350 INFO:teuthology.lock.ops:Updating [smithi075.front.sepia.ceph.com]: set os type and version on server 2022-07-06T13:08:30.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi075.front.sepia.ceph.com' 2022-07-06T13:08:30.593 INFO:teuthology.orchestra.run.smithi075.stdout:x86_64 2022-07-06T13:08:30.654 INFO:teuthology.orchestra.run.smithi075.stdout:NAME="CentOS Stream" 2022-07-06T13:08:30.655 INFO:teuthology.orchestra.run.smithi075.stdout:VERSION="8" 2022-07-06T13:08:30.655 INFO:teuthology.orchestra.run.smithi075.stdout:ID="centos" 2022-07-06T13:08:30.655 INFO:teuthology.orchestra.run.smithi075.stdout:ID_LIKE="rhel fedora" 2022-07-06T13:08:30.656 INFO:teuthology.orchestra.run.smithi075.stdout:VERSION_ID="8" 2022-07-06T13:08:30.656 INFO:teuthology.orchestra.run.smithi075.stdout:PLATFORM_ID="platform:el8" 2022-07-06T13:08:30.656 INFO:teuthology.orchestra.run.smithi075.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-06T13:08:30.657 INFO:teuthology.orchestra.run.smithi075.stdout:ANSI_COLOR="0;31" 2022-07-06T13:08:30.657 INFO:teuthology.orchestra.run.smithi075.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-06T13:08:30.658 INFO:teuthology.orchestra.run.smithi075.stdout:HOME_URL="https://centos.org/" 2022-07-06T13:08:30.658 INFO:teuthology.orchestra.run.smithi075.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-06T13:08:30.658 INFO:teuthology.orchestra.run.smithi075.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-06T13:08:30.658 INFO:teuthology.orchestra.run.smithi075.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-06T13:08:30.659 INFO:teuthology.lock.ops:Updating smithi075.front.sepia.ceph.com on lock server 2022-07-06T13:08:30.796 INFO:teuthology.dispatcher.supervisor:Running job 6917351 2022-07-06T13:08:30.796 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/bin/teuthology -v --owner scheduled_dparmar@teuthology --archive /home/teuthworker/archive/dparmar-2022-07-06_12:57:28-fs:upgrade-wip-dparmar-cephadm-55715-distro-default-smithi/6917351 --name dparmar-2022-07-06_12:57:28-fs:upgrade-wip-dparmar-cephadm-55715-distro-default-smithi --description fs:upgrade/upgrade_without_reducing_max_mds/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/dparmar-2022-07-06_12:57:28-fs:upgrade-wip-dparmar-cephadm-55715-distro-default-smithi/6917351/orig.config.yaml 2022-07-06T13:08:30.810 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dparmar-2022-07-06_12:57:28-fs:upgrade-wip-dparmar-cephadm-55715-distro-default-smithi/6917351 2022-07-06T13:08:30.812 INFO:teuthology.dispatcher.supervisor:Job PID: 28604 2022-07-06T13:08:30.813 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-07T01:09:39.651 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-07T01:09:40.560 INFO:teuthology.kill:Killing Pids: {28604} 2022-07-07T01:09:50.368 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2022-07-07T01:09:50.370 INFO:teuthology.task.internal:roles: ubuntu@smithi075.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2022-07-07T01:09:50.371 INFO:teuthology.misc:Compressing logs... 2022-07-07T01:09:50.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2022-07-07T01:09:51.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi075.front.sepia.ceph.com' 2022-07-07T01:09:51.849 INFO:teuthology.misc:Compressing logs... 2022-07-07T01:09:52.518 INFO:teuthology.misc:Compressing logs... 2022-07-07T01:10:00.748 INFO:teuthology.orchestra.run.smithi075.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-mon.smithi075.log: file size changed while zipping 2022-07-07T01:10:10.988 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-07-07T01:10:17.519 INFO:teuthology.orchestra.run.smithi075.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-07-07T01:10:32.682 INFO:teuthology.orchestra.run.smithi075.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-07-07T01:10:34.704 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-mon.smithi055.log: file size changed while zipping 2022-07-07T01:10:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-mgr.smithi055.rkdvid.log: file size changed while zipping 2022-07-07T01:10:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-07-07T01:11:15.862 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-07-07T01:11:16.000 INFO:teuthology.orchestra.run.smithi075.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-mds.cephfs.smithi075.vxpdxg.log: file size changed while zipping 2022-07-07T01:11:36.556 INFO:teuthology.orchestra.run.smithi075.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-07-07T01:12:20.728 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/4df20046-fd2e-11ec-842d-001a4aab830c/ceph-mds.cephfs.smithi055.wqzeda.log: file size changed while zipping 2022-07-07T01:13:53.796 INFO:teuthology.kill:No teuthology processes running 2022-07-07T01:13:56.195 INFO:teuthology.kill:Nuking machines: ['smithi055', 'smithi075'] 2022-07-07T01:13:58.576 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-07T01:13:58.577 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-07T01:13:58.578 INFO:teuthology.kill:2022-07-07 01:13:58,576.576 INFO:teuthology.nuke:targets: 2022-07-07T01:13:58.578 INFO:teuthology.kill: smithi055.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsnCqdeAE2ajAHLjib/H5emh34Rw+Ru/1iB+45d82NXvuVPBXeG6DFQehCGAZT1quHaOX3zXInhDMnSS1WpwHneGYA4UuIN9gmlqp3X9EXy3lWDcbagSaxlEVQifipKBb06dk0TFkMLEvTZj4KuO2ZHXt0VRDBczIb8xKJ3TvDVTXwaF2Ll+wON25z9pwVQ63pIsuSVlAWooCVbsUbecZOO0Wy2xll5iPl1OrrqasfZIMfb6rpTwb4mI59AeHpJl1PAml3yaZtPfoR9+mk0TbkJ7I0tjSLYeUtGx03XoghYHcAItwnuYuV1iHhygBfpoXUlveOKC2hqg5ixlQLTpgnEWPnilvtPahkIqbPPuhSc56sauFMghkm/G8VmvqbomlQcRQPfRWoqlPRPCtg3Ywz4DbFZ0EKKkaaD3rRAryMRcJ/lewMbMq5vWD35YMLa0lUQlo/gBSgB/hfKTBSAfRWFxWGSlFAzT7+BGb5X3gfBg4g1e47dIjvymXHofFeOFU= 2022-07-07T01:13:58.578 INFO:teuthology.kill: smithi075.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCeIieY2IKUQ5znDqmsscV1WHUqL4lCR2SO3ZNrGluK14lp0A8BEoNLEBJWdj7VoJXxiQ69jq8a7Tq/NetWsW/Mm69A0EplAQyhXpIwgiOtr9+e+zA7OkIzrIj4hoTVp65pqELpEKJE0L+ILgL14bsyMnSjrNPrgSAR+QozK6CACnQF4RRK3Tq+EMXwLTJLIRlbNgkCRIApE+ShX+8wHBjvQLZzAFdJBkY6t4+Q3GnGKrMtLsWdgY3osMPdXrisfOmbu6tg2luUj452sn62FX47aSK8RNcipx6w8Q27IdodkAdqgXu0R6AXQDfSsbO6jQEktOlXy37gDfWlzBqXDcrPlfX9hrZLXe5sh6/3FaaIA7CSwz+SxZSnfycQQaHEPh1qBDbmVN26i2Bqv0t6dqtSPvo6Xoc1RuUbqn9y/U0OtQImQcWaJ2uiR3O3bp7sdhKWHCvhOhHa3Ef9u7BiiIkazYS7mXWPlcXkMvaWks4UfjjHUDN2KTJGyLxOv0oublE= 2022-07-07T01:13:58.579 INFO:teuthology.kill:2022-07-07 01:13:58,578.578 INFO:teuthology.nuke:Checking targets against current locks 2022-07-07T01:13:58.655 INFO:teuthology.kill:2022-07-07 01:13:58,655.655 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-07T01:13:58.700 INFO:teuthology.kill:2022-07-07 01:13:58,700.700 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-07T01:13:58.785 INFO:teuthology.kill:2022-07-07 01:13:58,785.785 INFO:teuthology.orchestra.console:Power off smithi055 2022-07-07T01:13:58.869 INFO:teuthology.kill:2022-07-07 01:13:58,868.868 INFO:teuthology.orchestra.console:Power off smithi075 2022-07-07T01:14:07.235 INFO:teuthology.kill:2022-07-07 01:14:07,227.227 INFO:teuthology.orchestra.console:Power off for smithi055 completed 2022-07-07T01:14:07.441 INFO:teuthology.kill:2022-07-07 01:14:07,440.440 INFO:teuthology.lock.ops:unlocked: smithi055.front.sepia.ceph.com 2022-07-07T01:14:11.391 INFO:teuthology.kill:2022-07-07 01:14:11,385.385 INFO:teuthology.orchestra.console:Power off for smithi075 completed 2022-07-07T01:14:11.587 INFO:teuthology.kill:2022-07-07 01:14:11,583.583 INFO:teuthology.lock.ops:unlocked: smithi075.front.sepia.ceph.com 2022-07-07T01:16:12.175 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-07T01:16:12.384 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi055 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-07_00:16:01-fs-octopus-release-distro-default-smithi/6918010 2022-07-07T01:16:12.385 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi075 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-07_00:16:01-fs-octopus-release-distro-default-smithi/6918010