2021-11-18T19:07:11.002 INFO:root:teuthology version: 1.1.0-6cfddd5e
2021-11-18T19:07:11.125 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi104.front.sepia.ceph.com' reimaging
2021-11-18T19:07:11.126 INFO:teuthology.lock.ops:Updating [ubuntu@smithi104.front.sepia.ceph.com]: reset os type and version on server
2021-11-18T19:07:11.126 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2021-11-18T19:07:11.145 INFO:teuthology.lock.ops:Node 'ubuntu@smithi104.front.sepia.ceph.com' reimaging is complete
2021-11-18T19:07:11.146 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi123.front.sepia.ceph.com' reimaging
2021-11-18T19:07:11.146 INFO:teuthology.lock.ops:Updating [ubuntu@smithi123.front.sepia.ceph.com]: reset os type and version on server
2021-11-18T19:07:11.147 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2021-11-18T19:07:11.167 INFO:teuthology.lock.ops:Node 'ubuntu@smithi123.front.sepia.ceph.com' reimaging is complete
2021-11-18T19:07:11.361 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of centos 8.stream
2021-11-18T19:07:11.411 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of centos 8.stream
2021-11-18T19:07:11.776 INFO:teuthology.orchestra.console:Power off smithi104
2021-11-18T19:07:11.817 INFO:teuthology.orchestra.console:Power off smithi123
2021-11-18T19:07:11.928 INFO:teuthology.orchestra.console:Power off for smithi104 completed
2021-11-18T19:07:11.959 INFO:teuthology.orchestra.console:Power off for smithi123 completed
2021-11-18T19:07:12.030 INFO:teuthology.orchestra.console:Power on smithi104
2021-11-18T19:07:12.063 INFO:teuthology.orchestra.console:Power on smithi123
2021-11-18T19:07:20.404 INFO:teuthology.orchestra.console:Power on for smithi104 completed
2021-11-18T19:07:20.425 INFO:teuthology.orchestra.console:Power on for smithi123 completed
2021-11-18T19:07:20.506 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish
2021-11-18T19:07:20.526 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish
2021-11-18T19:09:51.657 ERROR:teuthology.orchestra.connection:Error authenticating with smithi104.front.sepia.ceph.com: Authentication failed.
2021-11-18T19:11:31.810 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-18T19:11:42.119 INFO:teuthology.provision.fog.smithi123:Node is ready
2021-11-18T19:11:42.138 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com
2021-11-18T19:11:42.194 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123
2021-11-18T19:11:42.536 INFO:teuthology.provision.fog.smithi123:Deploy complete!
2021-11-18T19:12:33.737 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-18T19:12:33.986 INFO:teuthology.provision.fog.smithi104:Node is ready
2021-11-18T19:12:34.042 INFO:teuthology.orchestra.run.smithi104.stdout:smithi104.front.sepia.ceph.com
2021-11-18T19:12:34.098 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.104 smithi104.front.sepia.ceph.com smithi104
2021-11-18T19:12:34.440 INFO:teuthology.provision.fog.smithi104:Deploy complete!
2021-11-18T19:12:34.584 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com
2021-11-18T19:12:34.585 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-18T19:12:34.609 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com
2021-11-18T19:12:34.609 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-18T19:12:34.630 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server
2021-11-18T19:12:34.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com'
2021-11-18T19:12:34.821 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64
2021-11-18T19:12:34.876 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream"
2021-11-18T19:12:34.876 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="8"
2021-11-18T19:12:34.877 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos"
2021-11-18T19:12:34.877 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora"
2021-11-18T19:12:34.877 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="8"
2021-11-18T19:12:34.877 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el8"
2021-11-18T19:12:34.878 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-18T19:12:34.878 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31"
2021-11-18T19:12:34.878 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-18T19:12:34.878 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/"
2021-11-18T19:12:34.878 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-18T19:12:34.879 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-18T19:12:34.879 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-18T19:12:34.880 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2021-11-18T19:12:34.898 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server
2021-11-18T19:12:34.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2021-11-18T19:12:35.141 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64
2021-11-18T19:12:35.199 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream"
2021-11-18T19:12:35.199 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8"
2021-11-18T19:12:35.199 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos"
2021-11-18T19:12:35.199 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora"
2021-11-18T19:12:35.200 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8"
2021-11-18T19:12:35.200 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8"
2021-11-18T19:12:35.200 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-18T19:12:35.200 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31"
2021-11-18T19:12:35.201 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-18T19:12:35.201 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/"
2021-11-18T19:12:35.201 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-18T19:12:35.201 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-18T19:12:35.202 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-18T19:12:35.202 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2021-11-18T19:12:35.299 INFO:teuthology.dispatcher.supervisor:Running job 6512218
2021-11-18T19:12:35.300 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2021-11-18_15:57:43-rados:cephadm-wip-guits-testing-2021-11-18-1333-pacific-distro-default-smithi/6512218 --name gabrioux-2021-11-18_15:57:43-rados:cephadm-wip-guits-testing-2021-11-18-1333-pacific-distro-default-smithi --description rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/gabrioux-2021-11-18_15:57:43-rados:cephadm-wip-guits-testing-2021-11-18-1333-pacific-distro-default-smithi/6512218/orig.config.yaml
2021-11-18T19:12:35.304 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2021-11-18_15:57:43-rados:cephadm-wip-guits-testing-2021-11-18-1333-pacific-distro-default-smithi/6512218
2021-11-18T19:12:35.305 INFO:teuthology.dispatcher.supervisor:Job PID: 22336
2021-11-18T19:12:35.305 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-19T07:12:56.533 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-19T07:12:56.744 INFO:teuthology.kill:Killing Pids: {22336}
2021-11-19T07:12:58.092 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-19T07:12:58.093 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-19T07:12:58.093 INFO:teuthology.misc:Compressing logs...
2021-11-19T07:12:58.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com'
2021-11-19T07:12:58.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2021-11-19T07:12:58.620 INFO:teuthology.misc:Compressing logs...
2021-11-19T07:12:58.649 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-19T07:12:58.686 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-19T07:12:58.717 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-19T07:12:58.718 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now
2021-11-19T07:12:58.753 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-19T07:12:58.753 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now
2021-11-19T07:12:58.754 INFO:teuthology.misc:Compressing logs...
2021-11-19T07:12:58.788 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists;	not overwritten
2021-11-19T07:12:58.798 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/dispatcher/supervisor.py", line 242, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/dispatcher/supervisor.py", line 300, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi104 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2021-11-19T07:12:58.878 INFO:teuthology.kill:No teuthology processes running
2021-11-19T07:13:00.134 INFO:teuthology.kill:Nuking machines: ['smithi104', 'smithi123']
2021-11-19T07:13:01.295 INFO:teuthology.kill:2021-11-19 07:13:01,295.295 INFO:teuthology.nuke:targets:
2021-11-19T07:13:01.296 INFO:teuthology.kill:  smithi104.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClntVwVyv6ch4abwoKQ8brapdysSG9whnIEIMcNDpuPCa27f5o/3HFdAeIX3ojj2pVSYlTCMnt3a1gK13fISmgn1FiX9MnBypMsbEfgT4oXuriN+v4R9g6ld8/Ed7ETzCyw8JQIYhqJqI+npoEiAUpcTVOovUU2vcu2XGtEQk94eGdkZTZi+ycrGz2a6h1LFrR8OpMgrGOuy+rGU9u9FblTtrxB5iY00/UdP/udJd01+rJBYmlwhtQUyaESuwksnVYW2Ep7LlJQ4itJhVXmzdN87Xn7F1vKnDko5DIBUr9hLVGHen9/KO1Zr/dBu3nTaJkQu+yui45geF+NGcHw5+VKcJlWIvyRPOyIrxzwhXWmdlNaPkmUEj62NpA7fDgSs3iT9ShAfS0OBWx5Xaed1KOCHJg5nf44vs4B01bZ4wtJz/Q5K+2l8DjHAUt9ugHlLz0IIdpm3tDKajP/xr9AALrR3cbH+mhjTN0xSPq77EFaVl34lgk1bvJVWM3wXA0QDE=
2021-11-19T07:13:01.296 INFO:teuthology.kill:  smithi123.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNLHQ6l7H+iWBN4Zia5WfhqyqdUxKzXhU2Yx8wKnmsFk2IeehnGPZtHpIA3jp0MAhPv9hP48/je3wrxDn9piWv8++SyLIsgcfV0Gn/JStxtnikjecICHe2byPRC9JEoDXP+e0PIGEPmBw+Sbs2w/WCI1yRDsPXrs1DSl9gUjSL3ZcPTnXvWsYW+kHu8CIBZCTmk2puD34/S18Jjmypdf2IV3R49D+qFxqrlpjGC07DwSwnBW5WO06wJ46BG+3/CTwNB4XFrfnjYjgHJscwl0ejM77ktkJew17lU3UnxpqUvxHpKzsM2BTUToqGaSE93RuM/T7FL3R/ppN4FagWX4s1+EaLMcCfO4RoOZxZsW7GbiI+Uha56Msdk9Eu8eu6fWUvFT/6VBAhFtqBO+tqoF0Hwz5U5oFkDwF1pzBHSTUO69/KPbwR5MV2j88AzjsCVzlRftR1Nk3nb3bchnsc0rV4rEcw5tcpUPxJfNyGcN/TdkDContPf/6KYzl/JoSG/O0=
2021-11-19T07:13:01.325 INFO:teuthology.kill:2021-11-19 07:13:01,324.324 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-19T07:13:01.328 INFO:teuthology.kill:2021-11-19 07:13:01,328.328 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-19T07:13:01.421 INFO:teuthology.kill:2021-11-19 07:13:01,421.421 INFO:teuthology.orchestra.console:Power off smithi104
2021-11-19T07:13:01.452 INFO:teuthology.kill:2021-11-19 07:13:01,451.451 INFO:teuthology.orchestra.console:Power off smithi123
2021-11-19T07:13:09.821 INFO:teuthology.kill:2021-11-19 07:13:09,820.820 INFO:teuthology.orchestra.console:Power off for smithi104 completed
2021-11-19T07:13:09.983 INFO:teuthology.kill:2021-11-19 07:13:09,981.981 INFO:teuthology.lock.ops:unlocked: smithi104.front.sepia.ceph.com
2021-11-19T07:13:13.946 INFO:teuthology.kill:2021-11-19 07:13:13,946.946 INFO:teuthology.orchestra.console:Power off for smithi123 completed
2021-11-19T07:13:14.102 INFO:teuthology.kill:2021-11-19 07:13:14,102.102 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com
2021-11-19T07:15:14.333 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-19T07:15:14.476 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi104 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-11-18_05:07:02-rgw-pacific-distro-default-smithi/6511361
2021-11-19T07:15:14.477 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi123 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-11-18_05:07:02-rgw-pacific-distro-default-smithi/6511361