2021-11-13T19:10:49.529 INFO:root:teuthology version: 1.1.0-d4737010
2021-11-13T19:10:49.716 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging
2021-11-13T19:10:49.717 INFO:teuthology.lock.ops:Updating [ubuntu@smithi134.front.sepia.ceph.com]: reset os type and version on server
2021-11-13T19:10:49.717 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server
2021-11-13T19:10:49.741 INFO:teuthology.lock.ops:Node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging is complete
2021-11-13T19:10:49.741 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi163.front.sepia.ceph.com' reimaging
2021-11-13T19:10:49.741 INFO:teuthology.lock.ops:Updating [ubuntu@smithi163.front.sepia.ceph.com]: reset os type and version on server
2021-11-13T19:10:49.742 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server
2021-11-13T19:10:49.764 INFO:teuthology.lock.ops:Node 'ubuntu@smithi163.front.sepia.ceph.com' reimaging is complete
2021-11-13T19:10:49.967 INFO:teuthology.provision.fog.smithi134:Scheduling deploy of centos 8.stream
2021-11-13T19:10:49.984 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of centos 8.stream
2021-11-13T19:10:50.309 INFO:teuthology.orchestra.console:Power off smithi134
2021-11-13T19:10:50.331 INFO:teuthology.orchestra.console:Power off smithi163
2021-11-13T19:10:58.699 INFO:teuthology.orchestra.console:Power off for smithi134 completed
2021-11-13T19:10:58.707 INFO:teuthology.orchestra.console:Power off for smithi163 completed
2021-11-13T19:10:58.802 INFO:teuthology.orchestra.console:Power on smithi134
2021-11-13T19:10:58.810 INFO:teuthology.orchestra.console:Power on smithi163
2021-11-13T19:11:07.179 INFO:teuthology.orchestra.console:Power on for smithi134 completed
2021-11-13T19:11:07.185 INFO:teuthology.orchestra.console:Power on for smithi163 completed
2021-11-13T19:11:07.281 INFO:teuthology.provision.fog.smithi134:Waiting for deploy to finish
2021-11-13T19:11:07.286 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish
2021-11-13T19:13:53.548 ERROR:teuthology.orchestra.connection:Error authenticating with smithi163.front.sepia.ceph.com: Authentication failed.
2021-11-13T19:15:56.773 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-13T19:15:57.050 INFO:teuthology.provision.fog.smithi134:Node is ready
2021-11-13T19:15:57.108 INFO:teuthology.orchestra.run.smithi134.stdout:smithi134.front.sepia.ceph.com
2021-11-13T19:15:57.168 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.15.134 smithi134.front.sepia.ceph.com smithi134
2021-11-13T19:15:57.503 INFO:teuthology.provision.fog.smithi134:Deploy complete!
2021-11-13T19:16:10.021 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-13T19:16:10.292 INFO:teuthology.provision.fog.smithi163:Node is ready
2021-11-13T19:16:10.309 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163.front.sepia.ceph.com
2021-11-13T19:16:10.365 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163
2021-11-13T19:16:10.706 INFO:teuthology.provision.fog.smithi163:Deploy complete!
2021-11-13T19:16:11.014 INFO:teuthology.lock.ops:Checking smithi134.front.sepia.ceph.com
2021-11-13T19:16:11.333 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-13T19:16:11.361 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com
2021-11-13T19:16:11.362 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-13T19:16:11.387 INFO:teuthology.lock.ops:Updating [smithi134.front.sepia.ceph.com]: set os type and version on server
2021-11-13T19:16:11.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com'
2021-11-13T19:16:11.652 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64
2021-11-13T19:16:11.713 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream"
2021-11-13T19:16:11.714 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8"
2021-11-13T19:16:11.714 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos"
2021-11-13T19:16:11.714 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora"
2021-11-13T19:16:11.714 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8"
2021-11-13T19:16:11.715 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8"
2021-11-13T19:16:11.715 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-13T19:16:11.715 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31"
2021-11-13T19:16:11.715 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-13T19:16:11.716 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/"
2021-11-13T19:16:11.716 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-13T19:16:11.716 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-13T19:16:11.716 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-13T19:16:11.717 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server
2021-11-13T19:16:11.756 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server
2021-11-13T19:16:11.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com'
2021-11-13T19:16:12.034 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64
2021-11-13T19:16:12.106 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream"
2021-11-13T19:16:12.107 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8"
2021-11-13T19:16:12.108 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos"
2021-11-13T19:16:12.108 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora"
2021-11-13T19:16:12.110 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8"
2021-11-13T19:16:12.111 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8"
2021-11-13T19:16:12.111 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-13T19:16:12.112 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31"
2021-11-13T19:16:12.112 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-13T19:16:12.112 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/"
2021-11-13T19:16:12.113 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-13T19:16:12.113 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-13T19:16:12.113 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-13T19:16:12.115 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server
2021-11-13T19:16:12.249 INFO:teuthology.dispatcher.supervisor:Running job 6501929
2021-11-13T19:16:12.250 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501929 --name yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501929/orig.config.yaml
2021-11-13T19:16:12.254 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501929
2021-11-13T19:16:12.255 INFO:teuthology.dispatcher.supervisor:Job PID: 9008
2021-11-13T19:16:12.255 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-14T07:16:32.996 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-14T07:16:33.193 INFO:teuthology.kill:Killing Pids: {9008}
2021-11-14T07:16:34.342 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-14T07:16:34.343 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-14T07:16:34.343 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:16:34.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com'
2021-11-14T07:16:34.530 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com'
2021-11-14T07:16:34.876 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:16:34.917 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-14T07:16:34.942 INFO:teuthology.orchestra.run.smithi163.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-14T07:16:34.988 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-14T07:16:34.988 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now
2021-11-14T07:16:35.013 INFO:teuthology.orchestra.run.smithi163.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-14T07:16:35.014 INFO:teuthology.orchestra.run.smithi163.stderr:tar: Error is not recoverable: exiting now
2021-11-14T07:16:35.015 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:16:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists;	not overwritten
2021-11-14T07:16:35.061 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/dispatcher/supervisor.py", line 242, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/dispatcher/supervisor.py", line 300, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi134 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2021-11-14T07:16:35.130 INFO:teuthology.kill:No teuthology processes running
2021-11-14T07:16:36.310 INFO:teuthology.kill:Nuking machines: ['smithi134', 'smithi163']
2021-11-14T07:16:37.306 INFO:teuthology.kill:2021-11-14 07:16:37,306.306 INFO:teuthology.nuke:targets:
2021-11-14T07:16:37.307 INFO:teuthology.kill:  smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc193mMVFuUu1RQFpNj7eKok05ONQtSt+aFWBkwubenoP05wAKFy8vlwSd8O5y8Blj/vuZnI6jSyfdI8rBQvmOyeUxjlYV7s4WLAzcQeGVim1MLnqLfymkaY7+nYPYoOR53Of3f8Gr53A1He4lSNkHrzju25aM5TDetE6OX5wd4MZeL3wz+HnILFqLYTotmlBiB087vCaxbIvLJ/ROvWbGW/+CC0vG35F0jH/R3AbXKhKdv04TEPQq96sr6OUs1jzTywEkxjz8OiyV6jR4jfNsic3qYdUxrDCpPPN8EttBD3kRoxz4LT/BWFumVOxAsR6uMf9ic1150FXY72L0RMUMNjbyc63b7+QBvVJhkRHAT7Sn+xgI4NrI+tLPuF8ni0TOUH58qzmLxgaE32lAQ+XjVnFVH9b08eI9lJdKVTAX2c68Nvwes38RjFwrz4vqXEvsM8IP5V8pSOwneUSXpIoMcaSdxmeDLNeM4f/negze8ImYh/3Met4Udbl51egE9cE=
2021-11-14T07:16:37.307 INFO:teuthology.kill:  smithi163.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDF8j6atRT3bcioe04yQnWTalloYj7/f5d+3UFfZyV84oKDJqG8U+mr1HSqasSb9hjyyjOlzAFyOnxQ7t0wDPUHiQffTY7Mfoa0GwzV1KPt8+RXFzzAAhCAmadvqMDmx7MtDOITbmm7nENezgkUrFCWp4xXh3MrXjHPIlvc58d3dn2p2rSieGYuM4kX/rJys+tyQQlsJenTdOOfeVv5qNuW2j+W1eRxSrR6TJSux2JZYLwimMu6s77YVdY/FwCW2t3DZx4Q8xBtekTTdEH8grWY2vxMRAzUqkz6DtONy0/HjVgE3eiPDdGtRmuOCU0kKaqLiPXK/Dt6+dbODkjjzyrqLT/yGRGDf6HpyLxtSjw2zr68IeTCHJzu0/+tcQOacZGY4iijPhmQP85oKaqpqkHXRldgTvm/0XpotxSrJR9PwdF0dt2nRQSZe0O7ETN40dEkTWTgNjSpyRNOx0h609K2ytLa6BSeT9tbQvNu+3F+xvD0BjAUXJA90sBPq83CYdk=
2021-11-14T07:16:37.334 INFO:teuthology.kill:2021-11-14 07:16:37,334.334 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-14T07:16:37.336 INFO:teuthology.kill:2021-11-14 07:16:37,335.335 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-14T07:16:37.412 INFO:teuthology.kill:2021-11-14 07:16:37,411.411 INFO:teuthology.orchestra.console:Power off smithi163
2021-11-14T07:16:37.425 INFO:teuthology.kill:2021-11-14 07:16:37,425.425 INFO:teuthology.orchestra.console:Power off smithi134
2021-11-14T07:16:49.893 INFO:teuthology.kill:2021-11-14 07:16:49,893.893 INFO:teuthology.orchestra.console:Power off for smithi163 completed
2021-11-14T07:16:49.898 INFO:teuthology.kill:2021-11-14 07:16:49,898.898 INFO:teuthology.orchestra.console:Power off for smithi134 completed
2021-11-14T07:16:50.038 INFO:teuthology.kill:2021-11-14 07:16:50,038.038 INFO:teuthology.lock.ops:unlocked: smithi163.front.sepia.ceph.com
2021-11-14T07:16:50.042 INFO:teuthology.kill:2021-11-14 07:16:50,042.042 INFO:teuthology.lock.ops:unlocked: smithi134.front.sepia.ceph.com
2021-11-14T07:18:50.238 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-14T07:18:50.350 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi134 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501786
2021-11-14T07:18:50.350 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi163 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501786