2022-07-04T18:35:00.480 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-07-04T18:35:00.613 INFO:teuthology.lock.ops:Start node 'smithi155.front.sepia.ceph.com' reimaging 2022-07-04T18:35:00.614 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: reset os type and version on server 2022-07-04T18:35:00.615 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2022-07-04T18:35:00.635 INFO:teuthology.lock.ops:Node 'smithi155.front.sepia.ceph.com' reimaging is complete 2022-07-04T18:35:00.635 INFO:teuthology.lock.ops:Start node 'smithi171.front.sepia.ceph.com' reimaging 2022-07-04T18:35:00.636 INFO:teuthology.lock.ops:Updating [smithi171.front.sepia.ceph.com]: reset os type and version on server 2022-07-04T18:35:00.636 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2022-07-04T18:35:00.657 INFO:teuthology.lock.ops:Node 'smithi171.front.sepia.ceph.com' reimaging is complete 2022-07-04T18:35:00.880 INFO:teuthology.provision.fog.smithi171:Scheduling deploy of centos 8.stream 2022-07-04T18:35:00.886 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of centos 8.stream 2022-07-04T18:35:01.245 INFO:teuthology.orchestra.console:Power off smithi155 2022-07-04T18:35:01.262 INFO:teuthology.orchestra.console:Power off smithi171 2022-07-04T18:35:09.622 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2022-07-04T18:35:09.724 INFO:teuthology.orchestra.console:Power on smithi155 2022-07-04T18:35:13.739 INFO:teuthology.orchestra.console:Power off for smithi171 completed 2022-07-04T18:35:13.841 INFO:teuthology.orchestra.console:Power on smithi171 2022-07-04T18:35:18.162 INFO:teuthology.orchestra.console:Power on for smithi155 completed 2022-07-04T18:35:18.216 INFO:teuthology.orchestra.console:Power on for smithi171 completed 2022-07-04T18:35:18.264 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2022-07-04T18:35:18.317 INFO:teuthology.provision.fog.smithi171:Waiting for deploy to finish 2022-07-04T18:39:39.509 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-04T18:40:04.830 INFO:teuthology.provision.fog.smithi155:Node is ready 2022-07-04T18:40:04.852 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2022-07-04T18:40:04.910 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2022-07-04T18:40:05.286 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2022-07-04T18:40:14.119 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-04T18:40:14.414 INFO:teuthology.provision.fog.smithi171:Node is ready 2022-07-04T18:40:14.433 INFO:teuthology.orchestra.run.smithi171.stdout:smithi171.front.sepia.ceph.com 2022-07-04T18:40:14.492 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.15.171 smithi171.front.sepia.ceph.com smithi171 2022-07-04T18:40:14.833 INFO:teuthology.provision.fog.smithi171:Deploy complete! 2022-07-04T18:40:15.093 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2022-07-04T18:40:15.095 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-04T18:40:15.118 INFO:teuthology.lock.ops:Checking smithi171.front.sepia.ceph.com 2022-07-04T18:40:15.118 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-04T18:40:15.139 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2022-07-04T18:40:15.140 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2022-07-04T18:40:15.344 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2022-07-04T18:40:15.403 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2022-07-04T18:40:15.404 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8" 2022-07-04T18:40:15.405 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2022-07-04T18:40:15.405 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2022-07-04T18:40:15.406 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8" 2022-07-04T18:40:15.406 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2022-07-04T18:40:15.407 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-04T18:40:15.407 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2022-07-04T18:40:15.407 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-04T18:40:15.407 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2022-07-04T18:40:15.408 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-04T18:40:15.408 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-04T18:40:15.408 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-04T18:40:15.409 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2022-07-04T18:40:15.431 INFO:teuthology.lock.ops:Updating [smithi171.front.sepia.ceph.com]: set os type and version on server 2022-07-04T18:40:15.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2022-07-04T18:40:15.647 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2022-07-04T18:40:15.705 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2022-07-04T18:40:15.705 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="8" 2022-07-04T18:40:15.706 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2022-07-04T18:40:15.706 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2022-07-04T18:40:15.706 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="8" 2022-07-04T18:40:15.707 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el8" 2022-07-04T18:40:15.707 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-04T18:40:15.707 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2022-07-04T18:40:15.707 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-04T18:40:15.708 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2022-07-04T18:40:15.708 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-04T18:40:15.708 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-04T18:40:15.708 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-04T18:40:15.709 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2022-07-04T18:40:15.867 INFO:teuthology.dispatcher.supervisor:Running job 6912711 2022-07-04T18:40:15.867 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/bin/teuthology -v --owner scheduled_dparmar@teuthology --archive /home/teuthworker/archive/dparmar-2022-07-04_17:09:23-fs:upgrade-wip-dparmar-cephadm-55715-testing-distro-default-smithi/6912711 --name dparmar-2022-07-04_17:09:23-fs:upgrade-wip-dparmar-cephadm-55715-testing-distro-default-smithi --description fs:upgrade/upgrade_without_reducing_max_mds/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/dparmar-2022-07-04_17:09:23-fs:upgrade-wip-dparmar-cephadm-55715-testing-distro-default-smithi/6912711/orig.config.yaml 2022-07-04T18:40:15.872 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dparmar-2022-07-04_17:09:23-fs:upgrade-wip-dparmar-cephadm-55715-testing-distro-default-smithi/6912711 2022-07-04T18:40:15.873 INFO:teuthology.dispatcher.supervisor:Job PID: 26604 2022-07-04T18:40:15.874 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-05T06:41:14.227 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-05T06:41:14.448 INFO:teuthology.kill:Killing Pids: {26604} 2022-07-05T06:41:16.047 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2022-07-05T06:41:16.049 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2022-07-05T06:41:16.050 INFO:teuthology.misc:Compressing logs... 2022-07-05T06:41:16.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2022-07-05T06:41:16.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2022-07-05T06:41:16.547 INFO:teuthology.misc:Compressing logs... 2022-07-05T06:41:16.843 INFO:teuthology.misc:Compressing logs... 2022-07-05T06:41:20.287 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-07-05T06:41:20.526 INFO:teuthology.orchestra.run.smithi171.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-07-05T06:41:23.396 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-07-05T06:41:24.024 INFO:teuthology.orchestra.run.smithi171.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-07-05T06:41:24.715 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-mgr.smithi155.nvfekg.log: file size changed while zipping 2022-07-05T06:41:26.480 INFO:teuthology.orchestra.run.smithi171.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-mon.smithi171.log: file size changed while zipping 2022-07-05T06:41:28.308 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-07-05T06:41:31.184 INFO:teuthology.orchestra.run.smithi171.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-07-05T06:41:34.250 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/a7aafd06-fbc9-11ec-842c-001a4aab830c/ceph-mon.smithi155.log: file size changed while zipping 2022-07-05T06:42:26.849 INFO:teuthology.kill:No teuthology processes running 2022-07-05T06:42:28.508 INFO:teuthology.kill:Nuking machines: ['smithi155', 'smithi171'] 2022-07-05T06:42:30.066 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-05T06:42:30.067 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-05T06:42:30.068 INFO:teuthology.kill:2022-07-05 06:42:30,066.066 INFO:teuthology.nuke:targets: 2022-07-05T06:42:30.068 INFO:teuthology.kill: smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBxIQbiu4XhcOl8fAv9MqekHITZD0bYyRJfMcFy0HTC4JR7KVMvx3Q1Yx0PlsjoAPA9g7eYYf76I1Jw0K582OUhdDu2Ttz0zRPa03NFKMJvJLL6jlkCXro5LJNN9bvUFUYxnYPsipOdGc5NKrYaKmwEMn+tpHf/M38KsLdby4jcPPSW7U/tRp/nBW9IDzfNohMWmuqgfeT1xPj3KV0Vq6sXY4sc96YbhKEIDGZheSv87EB+ICaBlpSVQ9utHCFq6VOlA+qa2Klh9ebc4AwDiQIY7p+K7mHGJTerdlQVHmVDLLcSGcK+7AJCLwEVx6hXUOVQ2cd/r9tod+HTgCvehLaSWaWftV9GMvrPsVeTvk7dfqDfdBSlWKl1a/JYr31RRylm1hMVeWIyxxbnND6o036Pi6IW0OKRPD4MBVfuW/zPVH2vvw4ZdX3hftBkU40zMO5ZXVjPf3ujiggGWy1Wt7e9QpwbGVAV0JdF/M69lO1aYk0oxZltaS0CoO9XciAEo0= 2022-07-05T06:42:30.068 INFO:teuthology.kill: smithi171.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQyxNtOJrAI4iU6FjGAsO5yo0U8YbA/6EiDEL6KklKFwpeiM0NMzgWuwl4fkalnYwR4LwhQSYkjvDRrTJsq80qXwLbx8OkJF5rSwRMeV6RO+YCnBfXXeyIQ9Aw26w8RBrP3ZgeVVU7qKfrshKf3Z5e/DXGVdiUZIkgT6xic3vL1VbsFL5spX3fe578CTBm+F1fsHcsPRf3hG8SGZZCjaArZENDqKOpkvAHhqqZNB7f9l+zduWNKI75s6Ki6PO7+3LPbbweWUCmHBGSsxxrty893LFT7bUQRGKFFCVblz9LZ8zJe0CH5/2Sq6qiPJpGcTg+fnebm/4lezoN0hoZfIJA5fYzEPGQGIKIMA5I1xlR3uwf7TX22blCZg3RAsC+fNKOXs2wcxMiZmwyyJ55U1k1f/EA7PRnJMTxpsEUkQrF0iLTpyl+97R0Hzd2hzR8FW7PsQLAToUk3uvTzZ9776GMgIsho46as/hDqFTAiGooy0nOgxXi+hiPxr9keelFFek= 2022-07-05T06:42:30.068 INFO:teuthology.kill:2022-07-05 06:42:30,067.067 INFO:teuthology.nuke:Checking targets against current locks 2022-07-05T06:42:30.136 INFO:teuthology.kill:2022-07-05 06:42:30,136.136 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-05T06:42:30.172 INFO:teuthology.kill:2022-07-05 06:42:30,171.171 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-05T06:42:30.252 INFO:teuthology.kill:2022-07-05 06:42:30,252.252 INFO:teuthology.orchestra.console:Power off smithi155 2022-07-05T06:42:30.301 INFO:teuthology.kill:2022-07-05 06:42:30,301.301 INFO:teuthology.orchestra.console:Power off smithi171 2022-07-05T06:42:42.767 INFO:teuthology.kill:2022-07-05 06:42:42,766.766 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2022-07-05T06:42:42.812 INFO:teuthology.kill:2022-07-05 06:42:42,812.812 INFO:teuthology.orchestra.console:Power off for smithi171 completed 2022-07-05T06:42:42.936 INFO:teuthology.kill:2022-07-05 06:42:42,934.934 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com 2022-07-05T06:42:42.983 INFO:teuthology.kill:2022-07-05 06:42:42,979.979 INFO:teuthology.lock.ops:unlocked: smithi171.front.sepia.ceph.com 2022-07-05T06:44:43.434 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-05T06:44:43.590 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi155 but it was locked by another job: /home/teuthworker/archive/dparmar-2022-07-05_05:21:58-fs:upgrade-wip-dparmar-cephadm-55715-testing-distro-default-smithi/6913732 2022-07-05T06:44:43.591 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi171 but it was locked by another job: /home/teuthworker/archive/dparmar-2022-07-05_05:21:58-fs:upgrade-wip-dparmar-cephadm-55715-testing-distro-default-smithi/6913732