2021-11-17T13:51:42.175 INFO:root:teuthology version: 1.1.0-6cfddd5e
2021-11-17T13:51:42.300 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi007.front.sepia.ceph.com' reimaging
2021-11-17T13:51:42.302 INFO:teuthology.lock.ops:Updating [ubuntu@smithi007.front.sepia.ceph.com]: reset os type and version on server
2021-11-17T13:51:42.304 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server
2021-11-17T13:51:42.323 INFO:teuthology.lock.ops:Node 'ubuntu@smithi007.front.sepia.ceph.com' reimaging is complete
2021-11-17T13:51:42.323 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi035.front.sepia.ceph.com' reimaging
2021-11-17T13:51:42.324 INFO:teuthology.lock.ops:Updating [ubuntu@smithi035.front.sepia.ceph.com]: reset os type and version on server
2021-11-17T13:51:42.325 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server
2021-11-17T13:51:42.347 INFO:teuthology.lock.ops:Node 'ubuntu@smithi035.front.sepia.ceph.com' reimaging is complete
2021-11-17T13:51:42.590 INFO:teuthology.provision.fog.smithi007:Scheduling deploy of centos 8.stream
2021-11-17T13:51:42.597 INFO:teuthology.provision.fog.smithi035:Scheduling deploy of centos 8.stream
2021-11-17T13:51:43.924 INFO:teuthology.orchestra.console:Power off smithi007
2021-11-17T13:51:44.065 INFO:teuthology.orchestra.console:Power off for smithi007 completed
2021-11-17T13:51:44.091 INFO:teuthology.orchestra.console:Power off smithi035
2021-11-17T13:51:44.166 INFO:teuthology.orchestra.console:Power on smithi007
2021-11-17T13:51:44.240 INFO:teuthology.orchestra.console:Power off for smithi035 completed
2021-11-17T13:51:44.341 INFO:teuthology.orchestra.console:Power on smithi035
2021-11-17T13:51:48.428 INFO:teuthology.orchestra.console:Power on for smithi007 completed
2021-11-17T13:51:48.530 INFO:teuthology.provision.fog.smithi007:Waiting for deploy to finish
2021-11-17T13:51:52.702 INFO:teuthology.orchestra.console:Power on for smithi035 completed
2021-11-17T13:51:52.803 INFO:teuthology.provision.fog.smithi035:Waiting for deploy to finish
2021-11-17T13:55:37.320 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-17T13:55:48.844 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-17T13:55:52.481 INFO:teuthology.provision.fog.smithi035:Node is ready
2021-11-17T13:55:52.503 INFO:teuthology.orchestra.run.smithi035.stdout:smithi035.front.sepia.ceph.com
2021-11-17T13:55:52.561 INFO:teuthology.orchestra.run.smithi035.stdout:172.21.15.35 smithi035.front.sepia.ceph.com smithi035
2021-11-17T13:55:52.923 INFO:teuthology.provision.fog.smithi035:Deploy complete!
2021-11-17T13:55:59.207 INFO:teuthology.provision.fog.smithi007:Node is ready
2021-11-17T13:55:59.228 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007.front.sepia.ceph.com
2021-11-17T13:55:59.287 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7 smithi007.front.sepia.ceph.com smithi007
2021-11-17T13:55:59.663 INFO:teuthology.provision.fog.smithi007:Deploy complete!
2021-11-17T13:55:59.847 INFO:teuthology.lock.ops:Checking smithi007.front.sepia.ceph.com
2021-11-17T13:55:59.848 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-17T13:55:59.868 INFO:teuthology.lock.ops:Checking smithi035.front.sepia.ceph.com
2021-11-17T13:55:59.868 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-17T13:55:59.885 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: set os type and version on server
2021-11-17T13:55:59.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com'
2021-11-17T13:56:00.123 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64
2021-11-17T13:56:00.178 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream"
2021-11-17T13:56:00.179 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8"
2021-11-17T13:56:00.179 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos"
2021-11-17T13:56:00.179 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora"
2021-11-17T13:56:00.179 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8"
2021-11-17T13:56:00.180 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8"
2021-11-17T13:56:00.180 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-17T13:56:00.180 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31"
2021-11-17T13:56:00.180 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-17T13:56:00.181 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/"
2021-11-17T13:56:00.181 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-17T13:56:00.181 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-17T13:56:00.181 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-17T13:56:00.183 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server
2021-11-17T13:56:00.200 INFO:teuthology.lock.ops:Updating [smithi035.front.sepia.ceph.com]: set os type and version on server
2021-11-17T13:56:00.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com'
2021-11-17T13:56:00.404 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64
2021-11-17T13:56:00.464 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="CentOS Stream"
2021-11-17T13:56:00.465 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="8"
2021-11-17T13:56:00.465 INFO:teuthology.orchestra.run.smithi035.stdout:ID="centos"
2021-11-17T13:56:00.466 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE="rhel fedora"
2021-11-17T13:56:00.466 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="8"
2021-11-17T13:56:00.466 INFO:teuthology.orchestra.run.smithi035.stdout:PLATFORM_ID="platform:el8"
2021-11-17T13:56:00.467 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-17T13:56:00.467 INFO:teuthology.orchestra.run.smithi035.stdout:ANSI_COLOR="0;31"
2021-11-17T13:56:00.468 INFO:teuthology.orchestra.run.smithi035.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-17T13:56:00.468 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://centos.org/"
2021-11-17T13:56:00.468 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-17T13:56:00.469 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-17T13:56:00.469 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-17T13:56:00.470 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server
2021-11-17T13:56:00.582 INFO:teuthology.dispatcher.supervisor:Running job 6509463
2021-11-17T13:56:00.583 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology -v --owner scheduled_swagner@teuthology --archive /home/teuthworker/archive/swagner-2021-11-17_12:55:30-rados:cephadm-wip-swagner2-testing-2021-11-17-1126-pacific-distro-default-smithi/6509463 --name swagner-2021-11-17_12:55:30-rados:cephadm-wip-swagner2-testing-2021-11-17-1126-pacific-distro-default-smithi --description rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/swagner-2021-11-17_12:55:30-rados:cephadm-wip-swagner2-testing-2021-11-17-1126-pacific-distro-default-smithi/6509463/orig.config.yaml
2021-11-17T13:56:00.587 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/swagner-2021-11-17_12:55:30-rados:cephadm-wip-swagner2-testing-2021-11-17-1126-pacific-distro-default-smithi/6509463
2021-11-17T13:56:00.588 INFO:teuthology.dispatcher.supervisor:Job PID: 5739
2021-11-17T13:56:00.588 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-18T01:56:55.531 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-18T01:56:55.746 INFO:teuthology.kill:Killing Pids: {5739}
2021-11-18T01:56:56.977 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-18T01:56:56.978 INFO:teuthology.task.internal:roles: ubuntu@smithi035.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-18T01:56:56.979 INFO:teuthology.misc:Compressing logs...
2021-11-18T01:56:56.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com'
2021-11-18T01:56:57.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com'
2021-11-18T01:56:57.553 INFO:teuthology.misc:Compressing logs...
2021-11-18T01:56:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-18T01:56:57.620 INFO:teuthology.orchestra.run.smithi035.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-18T01:56:57.655 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-18T01:56:57.656 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now
2021-11-18T01:56:57.691 INFO:teuthology.orchestra.run.smithi035.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-18T01:56:57.691 INFO:teuthology.orchestra.run.smithi035.stderr:tar: Error is not recoverable: exiting now
2021-11-18T01:56:57.692 INFO:teuthology.misc:Compressing logs...
2021-11-18T01:56:57.730 INFO:teuthology.orchestra.run.smithi007.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists;	not overwritten
2021-11-18T01:56:57.734 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/dispatcher/supervisor.py", line 242, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/dispatcher/supervisor.py", line 300, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi007 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2021-11-18T01:56:57.805 INFO:teuthology.kill:No teuthology processes running
2021-11-18T01:56:58.992 INFO:teuthology.kill:Nuking machines: ['smithi007', 'smithi035']
2021-11-18T01:56:59.984 INFO:teuthology.kill:2021-11-18 01:56:59,983.983 INFO:teuthology.nuke:targets:
2021-11-18T01:56:59.985 INFO:teuthology.kill:  smithi007.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoSqHlHQqPTgnNV8uKoYthSzKQVs4KjfA+DHBph7GPZ4GPplH+RglyowxFZ4cBQArDk99I4kAVWSPWVuTtIpbNel7gSKl7FxThVtkneqKmrM9gbeh7m0a/544Mvw9pBr6OI8nmAdup33R1pgznxKZUW5PgCOJw2Rn48Fkob/2/V+IgiM6gVLyxKXYnqYrs1ZD+jGpHvQCQ3fTAEv8I+MAk+AMLP1d+HyFcy6JTwcz4nVv+GqVc10qFChCkfqtypJi3fZZ/ZyBrU69e7CkhiUHKHJlbZ4LpwS979LPXnux4ZzxDcWXaMIKyZ0RnmGqiXQkiztqq4AuMiNFS0RA1T3gsIiFvY/0ZgmKR7qIlAFJuRaRpU9uEEq6oV/rm8lhaSBHd6vgqFiqoNJ3yIb6Oo0g5fmjRv5WDcHBbaI7f5ABXhSdk81M+ePCOh5Aj77s8Y9QgH2lqjybYg5eVp17YUb3gG57adOwMFHlVrDrVwG1S/S+F6ZNJGW0XrmmKNaQSuC0=
2021-11-18T01:56:59.985 INFO:teuthology.kill:  smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbE9vhL88Q/rN070p8s5vtYK9sC/PQS6JLE9634qDxihDL4EPAr6S3ysjeqGyFTUCKNQfYr/rJRI79e2Irz8HtO4NW7iQtRg1+SUeVXQ8BwXbbicPvXhl7o6Pb1aeYgjFpKFrxdCp89UQrfKeEqqPgome7lltiwSr7mjoeGDgCLTYzqreXYnRmSrGGNFEnj2xUuMlF5dDX3Nh1RZgysmfyRTr72WOAAytbyEKz4xrIITxa0+mUREkCJ9QqW1cCVnszwEoFjK95czKLSQjDQiHlz8Q1Yi/xrhKTBNV+C0sR4dhe4jTMa58fllf216oukrh/HwwQDbOZnfcDF7BWQUeIL1SFVlqcL/6jI+D/CaAUKMdRkA6aCRFf/fJX7oqTZWT4Q6cGJ1qxaFMgSsBbyhZxeI6V5i/A83V3muK9mu3GL5tbm2Nb0CgwGDDmNBRTVHaK2QwWGbQ3UStULok3Cm57osNpTjIKiE1UHKWoUZ3hiiAM+yo2usWOVdlkQACHYJk=
2021-11-18T01:57:00.010 INFO:teuthology.kill:2021-11-18 01:57:00,010.010 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-18T01:57:00.014 INFO:teuthology.kill:2021-11-18 01:57:00,013.013 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-18T01:57:00.093 INFO:teuthology.kill:2021-11-18 01:57:00,092.092 INFO:teuthology.orchestra.console:Power off smithi035
2021-11-18T01:57:00.107 INFO:teuthology.kill:2021-11-18 01:57:00,106.106 INFO:teuthology.orchestra.console:Power off smithi007
2021-11-18T01:57:08.461 INFO:teuthology.kill:2021-11-18 01:57:08,461.461 INFO:teuthology.orchestra.console:Power off for smithi035 completed
2021-11-18T01:57:08.470 INFO:teuthology.kill:2021-11-18 01:57:08,469.469 INFO:teuthology.orchestra.console:Power off for smithi007 completed
2021-11-18T01:57:08.611 INFO:teuthology.kill:2021-11-18 01:57:08,611.611 INFO:teuthology.lock.ops:unlocked: smithi035.front.sepia.ceph.com
2021-11-18T01:57:08.619 INFO:teuthology.kill:2021-11-18 01:57:08,619.619 INFO:teuthology.lock.ops:unlocked: smithi007.front.sepia.ceph.com
2021-11-18T01:59:08.958 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-18T01:59:09.074 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi007 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-18_00:45:42-rados-wip-yuri11-testing-2021-11-17-1514-pacific-distro-basic-smithi/6510841
2021-11-18T01:59:09.075 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi035 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-18_00:45:42-rados-wip-yuri11-testing-2021-11-17-1514-pacific-distro-basic-smithi/6510842