2021-11-13T19:03:56.597 INFO:root:teuthology version: 1.1.0-d4737010
2021-11-13T19:03:56.723 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi157.front.sepia.ceph.com' reimaging
2021-11-13T19:03:56.725 INFO:teuthology.lock.ops:Updating [ubuntu@smithi157.front.sepia.ceph.com]: reset os type and version on server
2021-11-13T19:03:56.726 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server
2021-11-13T19:03:56.745 INFO:teuthology.lock.ops:Node 'ubuntu@smithi157.front.sepia.ceph.com' reimaging is complete
2021-11-13T19:03:56.745 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi183.front.sepia.ceph.com' reimaging
2021-11-13T19:03:56.745 INFO:teuthology.lock.ops:Updating [ubuntu@smithi183.front.sepia.ceph.com]: reset os type and version on server
2021-11-13T19:03:56.746 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2021-11-13T19:03:56.766 INFO:teuthology.lock.ops:Node 'ubuntu@smithi183.front.sepia.ceph.com' reimaging is complete
2021-11-13T19:03:56.956 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of centos 8.stream
2021-11-13T19:03:56.967 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of centos 8.stream
2021-11-13T19:03:57.284 INFO:teuthology.orchestra.console:Power off smithi157
2021-11-13T19:03:57.301 INFO:teuthology.orchestra.console:Power off smithi183
2021-11-13T19:04:09.780 INFO:teuthology.orchestra.console:Power off for smithi157 completed
2021-11-13T19:04:09.784 INFO:teuthology.orchestra.console:Power off for smithi183 completed
2021-11-13T19:04:09.882 INFO:teuthology.orchestra.console:Power on smithi157
2021-11-13T19:04:09.890 INFO:teuthology.orchestra.console:Power on smithi183
2021-11-13T19:04:14.147 INFO:teuthology.orchestra.console:Power on for smithi183 completed
2021-11-13T19:04:14.249 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish
2021-11-13T19:04:18.252 INFO:teuthology.orchestra.console:Power on for smithi157 completed
2021-11-13T19:04:18.354 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish
2021-11-13T19:09:18.969 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-13T19:09:19.272 INFO:teuthology.provision.fog.smithi183:Node is ready
2021-11-13T19:09:19.290 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com
2021-11-13T19:09:19.367 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183
2021-11-13T19:09:19.717 INFO:teuthology.provision.fog.smithi183:Deploy complete!
2021-11-13T19:09:37.924 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-13T19:09:38.182 INFO:teuthology.provision.fog.smithi157:Node is ready
2021-11-13T19:09:38.240 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com
2021-11-13T19:09:38.299 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157
2021-11-13T19:09:38.636 INFO:teuthology.provision.fog.smithi157:Deploy complete!
2021-11-13T19:09:38.802 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com
2021-11-13T19:09:38.803 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-13T19:09:38.824 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com
2021-11-13T19:09:38.824 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-13T19:09:38.845 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server
2021-11-13T19:09:38.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com'
2021-11-13T19:09:39.034 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64
2021-11-13T19:09:39.088 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream"
2021-11-13T19:09:39.089 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8"
2021-11-13T19:09:39.089 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos"
2021-11-13T19:09:39.089 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora"
2021-11-13T19:09:39.089 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8"
2021-11-13T19:09:39.090 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8"
2021-11-13T19:09:39.090 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-13T19:09:39.090 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31"
2021-11-13T19:09:39.090 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-13T19:09:39.091 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/"
2021-11-13T19:09:39.091 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-13T19:09:39.091 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-13T19:09:39.093 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-13T19:09:39.094 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server
2021-11-13T19:09:39.113 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server
2021-11-13T19:09:39.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com'
2021-11-13T19:09:39.347 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64
2021-11-13T19:09:39.403 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream"
2021-11-13T19:09:39.403 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8"
2021-11-13T19:09:39.404 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos"
2021-11-13T19:09:39.404 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora"
2021-11-13T19:09:39.404 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8"
2021-11-13T19:09:39.405 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8"
2021-11-13T19:09:39.405 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-13T19:09:39.406 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31"
2021-11-13T19:09:39.406 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-13T19:09:39.407 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/"
2021-11-13T19:09:39.407 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-13T19:09:39.408 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-13T19:09:39.408 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-13T19:09:39.409 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2021-11-13T19:09:39.520 INFO:teuthology.dispatcher.supervisor:Running job 6501925
2021-11-13T19:09:39.521 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501925 --name yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501925/orig.config.yaml
2021-11-13T19:09:39.524 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501925
2021-11-13T19:09:39.525 INFO:teuthology.dispatcher.supervisor:Job PID: 8586
2021-11-13T19:09:39.525 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-14T07:10:34.275 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-14T07:10:34.623 INFO:teuthology.kill:Killing Pids: {8586}
2021-11-14T07:10:35.912 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-14T07:10:35.913 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-14T07:10:35.914 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:10:35.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com'
2021-11-14T07:10:36.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com'
2021-11-14T07:10:36.451 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:10:36.490 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-14T07:10:36.517 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-14T07:10:36.560 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-14T07:10:36.561 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now
2021-11-14T07:10:36.585 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-14T07:10:36.586 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now
2021-11-14T07:10:36.587 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:10:36.629 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists;	not overwritten
2021-11-14T07:10:36.631 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/dispatcher/supervisor.py", line 242, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/dispatcher/supervisor.py", line 300, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi157 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2021-11-14T07:10:36.704 INFO:teuthology.kill:No teuthology processes running
2021-11-14T07:10:37.849 INFO:teuthology.kill:Nuking machines: ['smithi157', 'smithi183']
2021-11-14T07:10:38.882 INFO:teuthology.kill:2021-11-14 07:10:38,882.882 INFO:teuthology.nuke:targets:
2021-11-14T07:10:38.883 INFO:teuthology.kill:  smithi157.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0bs6RPV+q2khxpWjF4s/u7eDrPu3yZ9eUsASmyJkGYWzKMQM2tiWy2scAH/j0DAuFOqatS3FUbCwkH3E8IBDR+x/iKyh7DOkdrfh1Bi+YuSMRrk47VQXiY/lV+0HHIzUzbFqCe0xvSMkYzbGSJe3ygX+fjKIbQWgebfq2EESLeSdeLytfRzE4dw7nQ7l+1X1n1OxLndxmeQXidqUC3a4Gh5pXO+oCC/fIgjKpK0CWfvnVHfMcxUP189K1diqJDXHuSyX7uCpJtrYbV7EmuR4YIQbhv30Y6olBhyTLJDF6eNMF2RVIT9dDazruO3nfKaC7zUFJTZykIyTR+0n5XyL55zZXFxWLl+8p7Q527SAdRInVLtEuB19zZJ2ygy3pd2tZEDjhAZ1ju/Hwei4QKgAUCKpXfuG4YFT6FFwYwvMWoND01Hesd8kVn/7j1E5Bcn+0bPKiXNy8DyZYK2HkKnI7QycVt1/IDXa4CJ4nJdzkrufVmkiheab9DzOwaqkqFzM=
2021-11-14T07:10:38.883 INFO:teuthology.kill:  smithi183.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHCW5X4Vhkg8n4+62IgYV2WNMJo3/f3O7YcFkVO+uZpK/GzJ8rVCyYruJ0cuFikzpbxOq4J3+juWq8uZFh9dm2vsl5+a5MvYeNw4Hmk9jYOU9eo54+d3YUaX5ZPg2zAKAqi6BRI/gUav9NXjWzIl+qkwo2Mf7DcK8dgczjz0K9YnMOoOhI8CV+o/6TymD9f3TNNUmj4USxD367VC9Mg6cpd8mhIkqGKJ6uJ2E52AV9bPCOYCY+LOFlNZKtxBJDtScrUJhfKfSYdnTiWc1KVucU/cF4PdlBMAuEPEiIo8CPe98B+/dv+X/8MS1ggezzI0zksSRA7egXfsTW6j6Y/xvaj9wePqRtC+6Je3EoqdPKWqLjQhYFgLGUTjUoI0bv6LYB+NJvBlmGdyiN4n7WBVB6KF5TobBb07arkYCW2oKFwoBB6SoBvHZzrBpDvoungUvJ+PbAqWUXISOa7u2B1+4ebBdVUN8IkOmyf2Ng9fF4Y4rXrmWUv9eIJX9E3he6r2k=
2021-11-14T07:10:38.909 INFO:teuthology.kill:2021-11-14 07:10:38,908.908 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-14T07:10:38.912 INFO:teuthology.kill:2021-11-14 07:10:38,912.912 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-14T07:10:38.986 INFO:teuthology.kill:2021-11-14 07:10:38,985.985 INFO:teuthology.orchestra.console:Power off smithi183
2021-11-14T07:10:39.002 INFO:teuthology.kill:2021-11-14 07:10:39,002.002 INFO:teuthology.orchestra.console:Power off smithi157
2021-11-14T07:10:47.367 INFO:teuthology.kill:2021-11-14 07:10:47,367.367 INFO:teuthology.orchestra.console:Power off for smithi183 completed
2021-11-14T07:10:47.514 INFO:teuthology.kill:2021-11-14 07:10:47,513.513 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com
2021-11-14T07:10:51.488 INFO:teuthology.kill:2021-11-14 07:10:51,488.488 INFO:teuthology.orchestra.console:Power off for smithi157 completed
2021-11-14T07:10:51.635 INFO:teuthology.kill:2021-11-14 07:10:51,634.634 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com
2021-11-14T07:12:51.945 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-14T07:12:52.058 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi157 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501779
2021-11-14T07:12:52.058 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi183 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501779