2021-11-08T20:05:13.887 INFO:root:teuthology version: 1.1.0-27954452
2021-11-08T20:05:14.011 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi042.front.sepia.ceph.com' reimaging
2021-11-08T20:05:14.013 INFO:teuthology.lock.ops:Updating [ubuntu@smithi042.front.sepia.ceph.com]: reset os type and version on server
2021-11-08T20:05:14.013 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server
2021-11-08T20:05:14.032 INFO:teuthology.lock.ops:Node 'ubuntu@smithi042.front.sepia.ceph.com' reimaging is complete
2021-11-08T20:05:14.032 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging
2021-11-08T20:05:14.033 INFO:teuthology.lock.ops:Updating [ubuntu@smithi170.front.sepia.ceph.com]: reset os type and version on server
2021-11-08T20:05:14.033 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2021-11-08T20:05:14.053 INFO:teuthology.lock.ops:Node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging is complete
2021-11-08T20:05:14.246 INFO:teuthology.provision.fog.smithi042:Scheduling deploy of centos 8.stream
2021-11-08T20:05:14.257 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 8.stream
2021-11-08T20:05:14.576 INFO:teuthology.orchestra.console:Power off smithi170
2021-11-08T20:05:14.616 INFO:teuthology.orchestra.console:Power off smithi042
2021-11-08T20:05:14.726 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2021-11-08T20:05:14.765 INFO:teuthology.orchestra.console:Power off for smithi042 completed
2021-11-08T20:05:14.829 INFO:teuthology.orchestra.console:Power on smithi170
2021-11-08T20:05:14.866 INFO:teuthology.orchestra.console:Power on smithi042
2021-11-08T20:05:19.089 INFO:teuthology.orchestra.console:Power on for smithi170 completed
2021-11-08T20:05:19.190 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish
2021-11-08T20:05:23.235 INFO:teuthology.orchestra.console:Power on for smithi042 completed
2021-11-08T20:05:23.338 INFO:teuthology.provision.fog.smithi042:Waiting for deploy to finish
2021-11-08T20:09:57.791 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-08T20:09:58.063 INFO:teuthology.provision.fog.smithi042:Node is ready
2021-11-08T20:09:58.080 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042.front.sepia.ceph.com
2021-11-08T20:09:58.137 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42 smithi042.front.sepia.ceph.com smithi042
2021-11-08T20:09:58.486 INFO:teuthology.provision.fog.smithi042:Deploy complete!
2021-11-08T20:10:23.867 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-08T20:10:24.175 INFO:teuthology.provision.fog.smithi170:Node is ready
2021-11-08T20:10:24.191 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com
2021-11-08T20:10:24.248 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170
2021-11-08T20:10:24.605 INFO:teuthology.provision.fog.smithi170:Deploy complete!
2021-11-08T20:10:24.754 INFO:teuthology.lock.ops:Checking smithi042.front.sepia.ceph.com
2021-11-08T20:10:24.755 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-08T20:10:24.776 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com
2021-11-08T20:10:24.776 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-08T20:10:24.794 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: set os type and version on server
2021-11-08T20:10:24.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com'
2021-11-08T20:10:25.041 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64
2021-11-08T20:10:25.098 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream"
2021-11-08T20:10:25.098 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="8"
2021-11-08T20:10:25.098 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos"
2021-11-08T20:10:25.099 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora"
2021-11-08T20:10:25.099 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="8"
2021-11-08T20:10:25.099 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el8"
2021-11-08T20:10:25.099 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-08T20:10:25.100 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31"
2021-11-08T20:10:25.100 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-08T20:10:25.100 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/"
2021-11-08T20:10:25.100 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-08T20:10:25.100 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-08T20:10:25.101 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-08T20:10:25.102 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server
2021-11-08T20:10:25.121 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server
2021-11-08T20:10:25.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2021-11-08T20:10:25.306 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64
2021-11-08T20:10:25.363 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream"
2021-11-08T20:10:25.363 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8"
2021-11-08T20:10:25.363 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos"
2021-11-08T20:10:25.363 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora"
2021-11-08T20:10:25.364 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8"
2021-11-08T20:10:25.364 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8"
2021-11-08T20:10:25.364 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-08T20:10:25.364 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31"
2021-11-08T20:10:25.365 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-08T20:10:25.365 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/"
2021-11-08T20:10:25.365 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-08T20:10:25.365 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-08T20:10:25.365 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-08T20:10:25.366 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2021-11-08T20:10:25.465 INFO:teuthology.dispatcher.supervisor:Running job 6491508
2021-11-08T20:10:25.465 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-11-08_19:29:02-orch:cephadm-master-distro-basic-smithi/6491508 --name sage-2021-11-08_19:29:02-orch:cephadm-master-distro-basic-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/sage-2021-11-08_19:29:02-orch:cephadm-master-distro-basic-smithi/6491508/orig.config.yaml
2021-11-08T20:10:25.468 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-11-08_19:29:02-orch:cephadm-master-distro-basic-smithi/6491508
2021-11-08T20:10:25.469 INFO:teuthology.dispatcher.supervisor:Job PID: 15803
2021-11-08T20:10:25.470 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-09T08:11:19.361 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-09T08:11:19.603 INFO:teuthology.kill:Killing Pids: {15803}
2021-11-09T08:11:20.840 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-09T08:11:20.841 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-09T08:11:20.842 INFO:teuthology.misc:Compressing logs...
2021-11-09T08:11:20.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com'
2021-11-09T08:11:21.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2021-11-09T08:11:21.388 INFO:teuthology.misc:Compressing logs...
2021-11-09T08:11:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-09T08:11:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-09T08:11:21.498 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-09T08:11:21.499 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now
2021-11-09T08:11:21.525 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-09T08:11:21.526 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now
2021-11-09T08:11:21.527 INFO:teuthology.misc:Compressing logs...
2021-11-09T08:11:21.569 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists;	not overwritten
2021-11-09T08:11:21.571 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/teuthology/dispatcher/supervisor.py", line 242, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/teuthology/dispatcher/supervisor.py", line 300, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi042 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2021-11-09T08:11:21.646 INFO:teuthology.kill:No teuthology processes running
2021-11-09T08:11:22.820 INFO:teuthology.kill:Nuking machines: ['smithi042', 'smithi170']
2021-11-09T08:11:23.795 INFO:teuthology.kill:2021-11-09 08:11:23,795.795 INFO:teuthology.nuke:targets:
2021-11-09T08:11:23.796 INFO:teuthology.kill:  smithi042.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsbAdVGG/UJ0j1o9YzM4RegSav077NVlBKP2syV+7q+C/7qFp2v70zxZFzQJkzFSBy/AOWEoNdmT86V2acno5+PMMuXqQqx170WPhz3nu2NL9acR6koamO6paK/QrJxFzrvzRiq/ZGto8xmHwXQBl9jt2MG0BQUp8mufEI1EtEPpRCnnBQ1RuicDbe83wrOWM7vwdqdadY1BkqffyxoVKlc9ik6nTKEs7A2Y2rmZgfkO6hrH2TOUx7rdUOfBblpb13MV6asQwC10D5YgZHuBtyKsbV8c6iLcNL8HTqk+YSbX0yVGkj7UfpRjxt+V/oY7WRy2IfPBDKX+dtXTDpcm68jLV+lNTGMerAI06CRyk7jKZjAhCea9rVldhUIm2rAwRHU1C8ZggE+L1SK2LYOrHajjjQOEWpauaYA/jVN9BuvCYmq+cOspFVaoE7krKKpD2/IHQbMCWNPrsg+jpu0FpXHe1PyrZSKJEYmZ5sBbbhFiTZRXRAJ152oSolA+5znDM=
2021-11-09T08:11:23.796 INFO:teuthology.kill:  smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfdUNmBBaEWwpHOfxPAX0ib1uDNxbKzQS8zDQHugYVv7SCbIfa9ljx2wl1npWRppr5/g+ObAl2+0Y3XeubVyvXN1okyFGZtGeGAXhwP6LD6LLwNzn5CqwfjY/ctERqCXvYJT3Hygm8rgAYP6TJ13mwJI8QKtZ8jZjaYEmvh+3iZ/ULQa4bUDvwXs+ZLU+niYdpsEmawG/9eVgD1rgazGc5oVmYiyGmd5sVEsbOuK2ureI3ubZVtY9wd+I2nxJSzSb/2Mhwh9BbVJ+zCHGyfABAlJPB6ZxSluuy22BXIBoc+5r6qXmvirpVGAQFLpHLrd5wyRQ67C7gsuWWN+J+skg2beEtXjcqzTedMFz+Vz8lWW3f43us1AKHOdZ7aAMW8zSQhTXZymJ7reQl7byeQ5zq0WTnxcIW8I1KKhnsBVrPhvzldrOj6qv0qOQ3bCGXE7FzPAuW111izWyPx1PlmDJvQQhRVLoLgPuIT0KXFQczSdZJzg/A4YU5ubaCr1PCD3k=
2021-11-09T08:11:23.827 INFO:teuthology.kill:2021-11-09 08:11:23,827.827 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-09T08:11:23.830 INFO:teuthology.kill:2021-11-09 08:11:23,830.830 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-09T08:11:23.914 INFO:teuthology.kill:2021-11-09 08:11:23,914.914 INFO:teuthology.orchestra.console:Power off smithi042
2021-11-09T08:11:23.934 INFO:teuthology.kill:2021-11-09 08:11:23,933.933 INFO:teuthology.orchestra.console:Power off smithi170
2021-11-09T08:11:32.299 INFO:teuthology.kill:2021-11-09 08:11:32,299.299 INFO:teuthology.orchestra.console:Power off for smithi042 completed
2021-11-09T08:11:32.306 INFO:teuthology.kill:2021-11-09 08:11:32,305.305 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2021-11-09T08:11:32.456 INFO:teuthology.kill:2021-11-09 08:11:32,456.456 INFO:teuthology.lock.ops:unlocked: smithi042.front.sepia.ceph.com
2021-11-09T08:11:32.460 INFO:teuthology.kill:2021-11-09 08:11:32,460.460 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com
2021-11-09T08:13:32.779 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-09T08:13:32.892 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi042 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-08_20:12:29-rados-wip-yuri4-testing-2021-11-08-1005-pacific-distro-basic-smithi/6491732
2021-11-09T08:13:32.892 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi170 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-08_20:12:29-rados-wip-yuri4-testing-2021-11-08-1005-pacific-distro-basic-smithi/6491731