2021-11-14T06:40:15.341 INFO:root:teuthology version: 1.1.0-d4737010
2021-11-14T06:40:15.505 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi064.front.sepia.ceph.com' reimaging
2021-11-14T06:40:15.507 INFO:teuthology.lock.ops:Updating [ubuntu@smithi064.front.sepia.ceph.com]: reset os type and version on server
2021-11-14T06:40:15.507 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server
2021-11-14T06:40:15.525 INFO:teuthology.lock.ops:Node 'ubuntu@smithi064.front.sepia.ceph.com' reimaging is complete
2021-11-14T06:40:15.526 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi171.front.sepia.ceph.com' reimaging
2021-11-14T06:40:15.526 INFO:teuthology.lock.ops:Updating [ubuntu@smithi171.front.sepia.ceph.com]: reset os type and version on server
2021-11-14T06:40:15.526 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server
2021-11-14T06:40:15.545 INFO:teuthology.lock.ops:Node 'ubuntu@smithi171.front.sepia.ceph.com' reimaging is complete
2021-11-14T06:40:15.720 INFO:teuthology.provision.fog.smithi064:Scheduling deploy of centos 8.stream
2021-11-14T06:40:15.749 INFO:teuthology.provision.fog.smithi171:Scheduling deploy of centos 8.stream
2021-11-14T06:40:16.054 INFO:teuthology.orchestra.console:Power off smithi171
2021-11-14T06:40:16.070 INFO:teuthology.orchestra.console:Power off smithi064
2021-11-14T06:40:24.426 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2021-11-14T06:40:24.431 INFO:teuthology.orchestra.console:Power off for smithi064 completed
2021-11-14T06:40:24.527 INFO:teuthology.orchestra.console:Power on smithi171
2021-11-14T06:40:24.536 INFO:teuthology.orchestra.console:Power on smithi064
2021-11-14T06:40:32.914 INFO:teuthology.orchestra.console:Power on for smithi064 completed
2021-11-14T06:40:32.974 INFO:teuthology.orchestra.console:Power on for smithi171 completed
2021-11-14T06:40:33.016 INFO:teuthology.provision.fog.smithi064:Waiting for deploy to finish
2021-11-14T06:40:33.076 INFO:teuthology.provision.fog.smithi171:Waiting for deploy to finish
2021-11-14T06:45:22.680 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-14T06:45:22.971 INFO:teuthology.provision.fog.smithi064:Node is ready
2021-11-14T06:45:22.989 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064.front.sepia.ceph.com
2021-11-14T06:45:23.048 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64 smithi064.front.sepia.ceph.com smithi064
2021-11-14T06:45:23.404 INFO:teuthology.provision.fog.smithi064:Deploy complete!
2021-11-14T06:45:37.668 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-14T06:45:37.946 INFO:teuthology.provision.fog.smithi171:Node is ready
2021-11-14T06:45:37.965 INFO:teuthology.orchestra.run.smithi171.stdout:smithi171.front.sepia.ceph.com
2021-11-14T06:45:38.023 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.15.171 smithi171.front.sepia.ceph.com smithi171
2021-11-14T06:45:38.383 INFO:teuthology.provision.fog.smithi171:Deploy complete!
2021-11-14T06:45:38.548 INFO:teuthology.lock.ops:Checking smithi064.front.sepia.ceph.com
2021-11-14T06:45:38.549 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-14T06:45:38.568 INFO:teuthology.lock.ops:Checking smithi171.front.sepia.ceph.com
2021-11-14T06:45:38.568 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-14T06:45:38.586 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: set os type and version on server
2021-11-14T06:45:38.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com'
2021-11-14T06:45:38.844 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64
2021-11-14T06:45:38.903 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream"
2021-11-14T06:45:38.903 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8"
2021-11-14T06:45:38.904 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos"
2021-11-14T06:45:38.904 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora"
2021-11-14T06:45:38.904 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8"
2021-11-14T06:45:38.904 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8"
2021-11-14T06:45:38.904 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-14T06:45:38.905 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31"
2021-11-14T06:45:38.905 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-14T06:45:38.905 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/"
2021-11-14T06:45:38.906 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-14T06:45:38.906 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-14T06:45:38.907 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-14T06:45:38.908 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server
2021-11-14T06:45:38.927 INFO:teuthology.lock.ops:Updating [smithi171.front.sepia.ceph.com]: set os type and version on server
2021-11-14T06:45:38.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com'
2021-11-14T06:45:39.121 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64
2021-11-14T06:45:39.178 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream"
2021-11-14T06:45:39.178 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="8"
2021-11-14T06:45:39.178 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos"
2021-11-14T06:45:39.179 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora"
2021-11-14T06:45:39.179 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="8"
2021-11-14T06:45:39.179 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el8"
2021-11-14T06:45:39.179 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-14T06:45:39.180 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31"
2021-11-14T06:45:39.180 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-14T06:45:39.180 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/"
2021-11-14T06:45:39.180 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-14T06:45:39.181 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-14T06:45:39.181 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-14T06:45:39.181 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server
2021-11-14T06:45:39.284 INFO:teuthology.dispatcher.supervisor:Running job 6501753
2021-11-14T06:45:39.285 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501753 --name yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501753/orig.config.yaml
2021-11-14T06:45:39.288 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501753
2021-11-14T06:45:39.289 INFO:teuthology.dispatcher.supervisor:Job PID: 22750
2021-11-14T06:45:39.289 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-14T18:46:33.763 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-14T18:46:33.959 INFO:teuthology.kill:Killing Pids: {22750}
2021-11-14T18:46:35.214 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-14T18:46:35.215 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-14T18:46:35.216 INFO:teuthology.misc:Compressing logs...
2021-11-14T18:46:35.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com'
2021-11-14T18:46:35.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com'
2021-11-14T18:46:35.753 INFO:teuthology.misc:Compressing logs...
2021-11-14T18:46:35.793 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-14T18:46:35.817 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-14T18:46:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-14T18:46:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now
2021-11-14T18:46:35.889 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-14T18:46:35.889 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now
2021-11-14T18:46:35.890 INFO:teuthology.misc:Compressing logs...
2021-11-14T18:46:35.934 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists;	not overwritten
2021-11-14T18:46:35.935 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/dispatcher/supervisor.py", line 242, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/dispatcher/supervisor.py", line 300, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi064 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2021-11-14T18:46:35.999 INFO:teuthology.kill:No teuthology processes running
2021-11-14T18:46:37.131 INFO:teuthology.kill:Nuking machines: ['smithi064', 'smithi171']
2021-11-14T18:46:38.092 INFO:teuthology.kill:2021-11-14 18:46:38,092.092 INFO:teuthology.nuke:targets:
2021-11-14T18:46:38.093 INFO:teuthology.kill:  smithi064.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCz6RzTa6SqcILMkpxvyfpR2AMfn+Pd1uhiPyg23C5xW4P01B8d4lnuFDTlMV6/EoSywusUQ6LtvT+SMXFFur5+Esu5o71qda6vPWTgZlOuOmsBZZ9tqTtv7mMGLNc7/MnBXK9BtrkdWv+XLVICKaWmY9XJCKi9EmPCvUIdVJ0hdGFd4lrMm0GEqdmK3/P91iE1hmtrta+roKFYq7pFwtp7wDvoOcZbALPuW73J37huamI6EN6uVsikxCFY8K1xvmM6ysFEgMZ47ddf+0jUJh7CcA07R8RHTrV/obUyBrOj4poJJe8pJyKJiMKjHcwMNdqlzwQqu05k5xnOo+TbOa66gFhq8jKP21ynSESHXuWRM6xXgW9AvNJ9MPQo/txdzq+0ha+6OA9yeSsF49Q1JbFZjiEaCFRJ3Sq97YzgRLzW7DBiq1gSJ2TkEYGvuZkZowwCvChFgsP6yDIrgy4zGpfKtnH+jLJdluYmnZeDPRL070WhA4E97ZcMG+e6tTVUSfE=
2021-11-14T18:46:38.094 INFO:teuthology.kill:  smithi171.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2RM+8p0SNnGXD0I9rsWBdy27FX+H10CNmquDXXJU72cajEwUZqLWSF281NyaPPPEMnSAPG3DO6DOCjRzXLtIfkhDmtVCkQ00iwyVX+HKNzf0npHcJpCHJ/M3DERApU8BNNsGDeEyJaDAVr1mJIHdH2TG1+pcfnuacYTiG9KQRp2meh56XicWCHGVgW+vG4xE6V/Ebf5+BQAHhjIKaHsyjQUW6SObf8ZieH04fm8sTVjfAPelmrqaW91DxZApjGk7tHWqrMW7mJmZWSIfaVscu8D/V6ka415wCZqLFDDYsC9lMoDYiDaw4ApfjVKljDHybVkAMcYTiSN7RQ2JUvm6ImkOebccsuYekRSYnI8k7z6pK4IqLWEGnIQcfL+1vVekR8dS/whRpLA+BVKmyrkqBxSD6VrfBoP/oCr4n4zQ1VrrbRQ852/UKVQrz95j2f5fUgwHC109LHZ1I0HNS8FbXh75tAwOH78++Tjw4mcTctCbLwaUNoh9QhWvR1slr5hU=
2021-11-14T18:46:38.119 INFO:teuthology.kill:2021-11-14 18:46:38,119.119 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-14T18:46:38.122 INFO:teuthology.kill:2021-11-14 18:46:38,122.122 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-14T18:46:38.199 INFO:teuthology.kill:2021-11-14 18:46:38,199.199 INFO:teuthology.orchestra.console:Power off smithi064
2021-11-14T18:46:38.214 INFO:teuthology.kill:2021-11-14 18:46:38,213.213 INFO:teuthology.orchestra.console:Power off smithi171
2021-11-14T18:46:46.575 INFO:teuthology.kill:2021-11-14 18:46:46,574.574 INFO:teuthology.orchestra.console:Power off for smithi064 completed
2021-11-14T18:46:46.580 INFO:teuthology.kill:2021-11-14 18:46:46,579.579 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2021-11-14T18:46:46.726 INFO:teuthology.kill:2021-11-14 18:46:46,725.725 INFO:teuthology.lock.ops:unlocked: smithi171.front.sepia.ceph.com
2021-11-14T18:46:46.729 INFO:teuthology.kill:2021-11-14 18:46:46,729.729 INFO:teuthology.lock.ops:unlocked: smithi064.front.sepia.ceph.com
2021-11-14T18:48:47.069 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-14T18:48:47.180 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi064 but it was locked by another job: /home/teuthworker/archive/matan-2021-11-10_12:24:34-rados-wip-matanb-crimson-clog-distro-basic-smithi/6496163
2021-11-14T18:48:47.181 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi171 but it was locked by another job: /home/teuthworker/archive/matan-2021-11-10_12:24:34-rados-wip-matanb-crimson-clog-distro-basic-smithi/6496163