2021-11-19T08:03:30.999 INFO:root:teuthology version: 1.1.0-6cfddd5e
2021-11-19T08:03:31.131 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging
2021-11-19T08:03:31.132 INFO:teuthology.lock.ops:Updating [ubuntu@smithi068.front.sepia.ceph.com]: reset os type and version on server
2021-11-19T08:03:31.133 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server
2021-11-19T08:03:31.151 INFO:teuthology.lock.ops:Node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging is complete
2021-11-19T08:03:31.152 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi090.front.sepia.ceph.com' reimaging
2021-11-19T08:03:31.152 INFO:teuthology.lock.ops:Updating [ubuntu@smithi090.front.sepia.ceph.com]: reset os type and version on server
2021-11-19T08:03:31.152 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2021-11-19T08:03:31.172 INFO:teuthology.lock.ops:Node 'ubuntu@smithi090.front.sepia.ceph.com' reimaging is complete
2021-11-19T08:03:31.359 INFO:teuthology.provision.fog.smithi068:Scheduling deploy of centos 8.stream
2021-11-19T08:03:31.379 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of centos 8.stream
2021-11-19T08:03:31.724 INFO:teuthology.orchestra.console:Power off smithi068
2021-11-19T08:03:31.740 INFO:teuthology.orchestra.console:Power off smithi090
2021-11-19T08:03:40.104 INFO:teuthology.orchestra.console:Power off for smithi090 completed
2021-11-19T08:03:40.205 INFO:teuthology.orchestra.console:Power on smithi090
2021-11-19T08:03:44.208 INFO:teuthology.orchestra.console:Power off for smithi068 completed
2021-11-19T08:03:44.313 INFO:teuthology.orchestra.console:Power on smithi068
2021-11-19T08:03:48.567 INFO:teuthology.orchestra.console:Power on for smithi068 completed
2021-11-19T08:03:48.668 INFO:teuthology.provision.fog.smithi068:Waiting for deploy to finish
2021-11-19T08:03:48.685 INFO:teuthology.orchestra.console:Power on for smithi090 completed
2021-11-19T08:03:48.787 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish
2021-11-19T08:08:54.041 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-19T08:08:54.299 INFO:teuthology.provision.fog.smithi068:Node is ready
2021-11-19T08:08:54.357 INFO:teuthology.orchestra.run.smithi068.stdout:smithi068.front.sepia.ceph.com
2021-11-19T08:08:54.416 INFO:teuthology.orchestra.run.smithi068.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068
2021-11-19T08:08:54.771 INFO:teuthology.provision.fog.smithi068:Deploy complete!
2021-11-19T08:09:09.082 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-19T08:09:09.424 INFO:teuthology.provision.fog.smithi090:Node is ready
2021-11-19T08:09:09.440 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090.front.sepia.ceph.com
2021-11-19T08:09:09.501 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90 smithi090.front.sepia.ceph.com smithi090
2021-11-19T08:09:09.861 INFO:teuthology.provision.fog.smithi090:Deploy complete!
2021-11-19T08:09:10.007 INFO:teuthology.lock.ops:Checking smithi068.front.sepia.ceph.com
2021-11-19T08:09:10.008 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-19T08:09:10.027 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com
2021-11-19T08:09:10.028 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-19T08:09:10.046 INFO:teuthology.lock.ops:Updating [smithi068.front.sepia.ceph.com]: set os type and version on server
2021-11-19T08:09:10.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com'
2021-11-19T08:09:10.293 INFO:teuthology.orchestra.run.smithi068.stdout:x86_64
2021-11-19T08:09:10.349 INFO:teuthology.orchestra.run.smithi068.stdout:NAME="CentOS Stream"
2021-11-19T08:09:10.350 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION="8"
2021-11-19T08:09:10.350 INFO:teuthology.orchestra.run.smithi068.stdout:ID="centos"
2021-11-19T08:09:10.350 INFO:teuthology.orchestra.run.smithi068.stdout:ID_LIKE="rhel fedora"
2021-11-19T08:09:10.351 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_ID="8"
2021-11-19T08:09:10.351 INFO:teuthology.orchestra.run.smithi068.stdout:PLATFORM_ID="platform:el8"
2021-11-19T08:09:10.351 INFO:teuthology.orchestra.run.smithi068.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-19T08:09:10.351 INFO:teuthology.orchestra.run.smithi068.stdout:ANSI_COLOR="0;31"
2021-11-19T08:09:10.352 INFO:teuthology.orchestra.run.smithi068.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-19T08:09:10.352 INFO:teuthology.orchestra.run.smithi068.stdout:HOME_URL="https://centos.org/"
2021-11-19T08:09:10.352 INFO:teuthology.orchestra.run.smithi068.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-19T08:09:10.352 INFO:teuthology.orchestra.run.smithi068.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-19T08:09:10.353 INFO:teuthology.orchestra.run.smithi068.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-19T08:09:10.354 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server
2021-11-19T08:09:10.372 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server
2021-11-19T08:09:10.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2021-11-19T08:09:10.564 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64
2021-11-19T08:09:10.624 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream"
2021-11-19T08:09:10.624 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="8"
2021-11-19T08:09:10.625 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos"
2021-11-19T08:09:10.625 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora"
2021-11-19T08:09:10.625 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="8"
2021-11-19T08:09:10.625 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el8"
2021-11-19T08:09:10.626 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-19T08:09:10.626 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2021-11-19T08:09:10.626 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-19T08:09:10.626 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/"
2021-11-19T08:09:10.627 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-19T08:09:10.627 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-19T08:09:10.627 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-19T08:09:10.628 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2021-11-19T08:09:10.736 INFO:teuthology.dispatcher.supervisor:Running job 6513267
2021-11-19T08:09:10.737 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2021-11-19_08:00:30-rados:cephadm-wip-guits-testing-2021-11-18-1333-pacific-distro-default-smithi/6513267 --name gabrioux-2021-11-19_08:00:30-rados:cephadm-wip-guits-testing-2021-11-18-1333-pacific-distro-default-smithi --description rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/gabrioux-2021-11-19_08:00:30-rados:cephadm-wip-guits-testing-2021-11-18-1333-pacific-distro-default-smithi/6513267/orig.config.yaml
2021-11-19T08:09:10.741 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2021-11-19_08:00:30-rados:cephadm-wip-guits-testing-2021-11-18-1333-pacific-distro-default-smithi/6513267
2021-11-19T08:09:10.742 INFO:teuthology.dispatcher.supervisor:Job PID: 2588
2021-11-19T08:09:10.742 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-19T20:10:05.112 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-19T20:10:05.429 INFO:teuthology.kill:Killing Pids: {2588}
2021-11-19T20:10:06.641 INFO:teuthology.task.internal:roles: ubuntu@smithi068.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-19T20:10:06.642 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-19T20:10:06.642 INFO:teuthology.misc:Compressing logs...
2021-11-19T20:10:06.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com'
2021-11-19T20:10:06.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2021-11-19T20:10:07.178 INFO:teuthology.misc:Compressing logs...
2021-11-19T20:10:07.217 INFO:teuthology.orchestra.run.smithi068.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-19T20:10:07.243 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-19T20:10:07.289 INFO:teuthology.orchestra.run.smithi068.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-19T20:10:07.289 INFO:teuthology.orchestra.run.smithi068.stderr:tar: Error is not recoverable: exiting now
2021-11-19T20:10:07.314 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-19T20:10:07.315 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now
2021-11-19T20:10:07.316 INFO:teuthology.misc:Compressing logs...
2021-11-19T20:10:07.358 INFO:teuthology.orchestra.run.smithi068.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists;	not overwritten
2021-11-19T20:10:07.360 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/dispatcher/supervisor.py", line 242, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/dispatcher/supervisor.py", line 300, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi068 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2021-11-19T20:10:07.442 INFO:teuthology.kill:No teuthology processes running
2021-11-19T20:10:08.660 INFO:teuthology.kill:Nuking machines: ['smithi068', 'smithi090']
2021-11-19T20:10:09.657 INFO:teuthology.kill:2021-11-19 20:10:09,657.657 INFO:teuthology.nuke:targets:
2021-11-19T20:10:09.658 INFO:teuthology.kill:  smithi068.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZQMR4F59R+MBhHEAFCBy1WM9m4B8038xkkqeLDBPArgjgxIJkvYZObMa2juQfWWmgMvN4peIikLAagbaVDEP58gUekSC5nSzp/cWVMfHkTokKc1zSwZoQfATv4iNCZARKcSfpuAtEKNHtZOGMdc/cRSmEhD8SJ/B4gGtYr9SQWE96SI3waazUP56JZX6+RA1rvlWQn+aDmNvLnXr2fMxs/xDdSzvOjzW8YOr2jVeAyN0epSUvo5aFtZ3i7CTRDzpLRbvMTGwP3aOId+rED6US/EhB74vkmgbk3cxVcQvdDSItyyeoP4MUDZwJn7m9zJ5QDWQhP4S/2P++oiTPzdDnNtu4v4QNZ2RPr9hNftMMaCdseCosjiQCNmqHhj1T+pwkBQ/iS0GbTWV8J5r8dEd4HNrVM0JZP918FUzkn1nXrK/Py5qkJr0mRxzmYUu0mdW0iT2CWsvwqPK8nRpAHxXS3DlfhYVDNBjOUBBKSn4SzSdVavgfkd4B0a7u9NNflSE=
2021-11-19T20:10:09.659 INFO:teuthology.kill:  smithi090.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE1Wx/Uya+hjJdeAFjotymXXvh/OMQYPwr5VRlFDFwvCi46kRuFpvnk+d8EmmJoLqzBwZCNwvLoaBFifLMCgZvAyPIBbhRs3cuYwbzypZYKYFN/BTc+sJJAMjNSxHsD0jc0+cm3XZRPdPoyg19iDnkFBLA5nwxK17XCv9P3Hzr31kC3xg/b5tvnPL79WXncaThtXJz6fMKIwKxMgxWUDbMqslT/pANWKBcpbFXR6nv8qrXD1XRRwVUncIxlXrfFkmR4kPPv6EM87mRxJx+uUmosyjqXtE5zAFXV3vg8dt4/Loy/kONZPlmu7h3/Wxkbe8YWvzhWzXIESeYbMUb8vgswo+k0v5nFb+4+C55kcEybHk1dWcfkvbwypEbFqXyJiIftkgLvRC+HEwPUSSLDoL2BZjmqZcptrb7aS1TxuD4i/FtYtuHAgFJpAIhpYVh6LXzzrDpgWYRXctSyTbget0s5VSKj2tINvYBcs0nSrjzOaxGqmMCj2z1qc1Aa5s2vKc=
2021-11-19T20:10:09.684 INFO:teuthology.kill:2021-11-19 20:10:09,684.684 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-19T20:10:09.687 INFO:teuthology.kill:2021-11-19 20:10:09,687.687 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-19T20:10:09.769 INFO:teuthology.kill:2021-11-19 20:10:09,769.769 INFO:teuthology.orchestra.console:Power off smithi068
2021-11-19T20:10:09.789 INFO:teuthology.kill:2021-11-19 20:10:09,788.788 INFO:teuthology.orchestra.console:Power off smithi090
2021-11-19T20:10:18.154 INFO:teuthology.kill:2021-11-19 20:10:18,153.153 INFO:teuthology.orchestra.console:Power off for smithi090 completed
2021-11-19T20:10:18.302 INFO:teuthology.kill:2021-11-19 20:10:18,301.301 INFO:teuthology.lock.ops:unlocked: smithi090.front.sepia.ceph.com
2021-11-19T20:10:22.261 INFO:teuthology.kill:2021-11-19 20:10:22,260.260 INFO:teuthology.orchestra.console:Power off for smithi068 completed
2021-11-19T20:10:22.411 INFO:teuthology.kill:2021-11-19 20:10:22,411.411 INFO:teuthology.lock.ops:unlocked: smithi068.front.sepia.ceph.com
2021-11-19T20:12:22.727 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-19T20:12:22.901 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi068 but it was locked by another job: /home/teuthworker/archive/ksirivad-2021-11-19_19:14:07-rados-wip-autoscale-profile-scale-up-default-distro-basic-smithi/6514269
2021-11-19T20:12:22.902 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi090 but it was locked by another job: /home/teuthworker/archive/ksirivad-2021-11-19_19:14:07-rados-wip-autoscale-profile-scale-up-default-distro-basic-smithi/6514271