2021-11-13T19:14:32.104 INFO:root:teuthology version: 1.1.0-d4737010
2021-11-13T19:14:32.234 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi096.front.sepia.ceph.com' reimaging
2021-11-13T19:14:32.235 INFO:teuthology.lock.ops:Updating [ubuntu@smithi096.front.sepia.ceph.com]: reset os type and version on server
2021-11-13T19:14:32.236 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server
2021-11-13T19:14:32.255 INFO:teuthology.lock.ops:Node 'ubuntu@smithi096.front.sepia.ceph.com' reimaging is complete
2021-11-13T19:14:32.255 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi105.front.sepia.ceph.com' reimaging
2021-11-13T19:14:32.255 INFO:teuthology.lock.ops:Updating [ubuntu@smithi105.front.sepia.ceph.com]: reset os type and version on server
2021-11-13T19:14:32.256 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2021-11-13T19:14:32.276 INFO:teuthology.lock.ops:Node 'ubuntu@smithi105.front.sepia.ceph.com' reimaging is complete
2021-11-13T19:14:32.487 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of centos 8.stream
2021-11-13T19:14:32.491 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of centos 8.stream
2021-11-13T19:14:32.867 INFO:teuthology.orchestra.console:Power off smithi105
2021-11-13T19:14:32.882 INFO:teuthology.orchestra.console:Power off smithi096
2021-11-13T19:14:33.013 INFO:teuthology.orchestra.console:Power off for smithi105 completed
2021-11-13T19:14:33.023 INFO:teuthology.orchestra.console:Power off for smithi096 completed
2021-11-13T19:14:33.116 INFO:teuthology.orchestra.console:Power on smithi105
2021-11-13T19:14:33.124 INFO:teuthology.orchestra.console:Power on smithi096
2021-11-13T19:14:37.370 INFO:teuthology.orchestra.console:Power on for smithi105 completed
2021-11-13T19:14:37.472 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish
2021-11-13T19:14:41.490 INFO:teuthology.orchestra.console:Power on for smithi096 completed
2021-11-13T19:14:41.591 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish
2021-11-13T19:18:53.645 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-13T19:19:09.005 INFO:teuthology.provision.fog.smithi096:Node is ready
2021-11-13T19:19:09.026 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com
2021-11-13T19:19:09.085 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096
2021-11-13T19:19:09.435 INFO:teuthology.provision.fog.smithi096:Deploy complete!
2021-11-13T19:19:42.114 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-13T19:19:42.406 INFO:teuthology.provision.fog.smithi105:Node is ready
2021-11-13T19:19:42.422 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105.front.sepia.ceph.com
2021-11-13T19:19:42.480 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105 smithi105.front.sepia.ceph.com smithi105
2021-11-13T19:19:42.829 INFO:teuthology.provision.fog.smithi105:Deploy complete!
2021-11-13T19:19:42.972 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com
2021-11-13T19:19:42.974 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-13T19:19:42.993 INFO:teuthology.lock.ops:Checking smithi105.front.sepia.ceph.com
2021-11-13T19:19:42.993 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-13T19:19:43.012 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server
2021-11-13T19:19:43.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com'
2021-11-13T19:19:43.295 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64
2021-11-13T19:19:43.351 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream"
2021-11-13T19:19:43.352 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8"
2021-11-13T19:19:43.352 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos"
2021-11-13T19:19:43.352 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora"
2021-11-13T19:19:43.353 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8"
2021-11-13T19:19:43.353 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8"
2021-11-13T19:19:43.353 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-13T19:19:43.353 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31"
2021-11-13T19:19:43.353 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-13T19:19:43.354 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/"
2021-11-13T19:19:43.354 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-13T19:19:43.354 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-13T19:19:43.354 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-13T19:19:43.356 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server
2021-11-13T19:19:43.373 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: set os type and version on server
2021-11-13T19:19:43.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com'
2021-11-13T19:19:43.562 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64
2021-11-13T19:19:43.619 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream"
2021-11-13T19:19:43.619 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8"
2021-11-13T19:19:43.619 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos"
2021-11-13T19:19:43.620 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora"
2021-11-13T19:19:43.620 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8"
2021-11-13T19:19:43.620 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8"
2021-11-13T19:19:43.620 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 8"
2021-11-13T19:19:43.621 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31"
2021-11-13T19:19:43.621 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-13T19:19:43.621 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/"
2021-11-13T19:19:43.621 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-11-13T19:19:43.622 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-11-13T19:19:43.622 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-11-13T19:19:43.623 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2021-11-13T19:19:43.746 INFO:teuthology.dispatcher.supervisor:Running job 6501935
2021-11-13T19:19:43.747 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501935 --name yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501935/orig.config.yaml
2021-11-13T19:19:43.750 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501935
2021-11-13T19:19:43.751 INFO:teuthology.dispatcher.supervisor:Job PID: 18373
2021-11-13T19:19:43.752 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-14T07:20:38.152 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-14T07:20:38.382 INFO:teuthology.kill:Killing Pids: {18373}
2021-11-14T07:20:39.591 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-14T07:20:39.592 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-14T07:20:39.593 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:20:39.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com'
2021-11-14T07:20:39.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com'
2021-11-14T07:20:40.130 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:20:40.167 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-14T07:20:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2021-11-14T07:20:40.247 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-14T07:20:40.247 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now
2021-11-14T07:20:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2021-11-14T07:20:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now
2021-11-14T07:20:40.274 INFO:teuthology.misc:Compressing logs...
2021-11-14T07:20:40.315 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists;	not overwritten
2021-11-14T07:20:40.317 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/dispatcher/supervisor.py", line 242, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/dispatcher/supervisor.py", line 300, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi096 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2021-11-14T07:20:40.387 INFO:teuthology.kill:No teuthology processes running
2021-11-14T07:20:41.605 INFO:teuthology.kill:Nuking machines: ['smithi096', 'smithi105']
2021-11-14T07:20:42.581 INFO:teuthology.kill:2021-11-14 07:20:42,580.580 INFO:teuthology.nuke:targets:
2021-11-14T07:20:42.582 INFO:teuthology.kill:  smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0e+Q3XqfAbLuK0Xw7GQ/fyznY5pl2rBRmeFYPw9+k7k6mIRlnCREafslmmct+oXhgdzLKVw6Eo4BXoDQsoJvkRoiTGspBFzeoWA24KmTvqQUQlUlb0FfjdIwuvqPV06akeJd3mxuIjNbJ6FGsJpvVmb4fazUZMejqFK3yFYu2yB0+45vdmBFoZ+yjpz7gbi+eN0950ZPwQV6wcBjePCzrNu7vvwfMzDulT6moYcKMUKiHN5It1Hf6tSXjwC09qlx39Ff0JVfPm7104KSTarh3E1fcSW1Zd2GIdWBsd3nDwMthmIcsqBUvWLzhHZlSanBNcDmnCPUUrVqGW6TxyCf61VK8cielNrMzUhrnY8il5wdLIxzPBLOE7Gg61uKOixcYi5b4u6IxHsNBGVRF3MbevDeVxNdP1KwNGIiptmmm1wlPZJ0H3urXxvH/8pD2k5dkoX+Ns3i0cHsJA49d3oZ54toic5raNBzJOza19gF5XcYI23snTm3NiudzOjtsvzM=
2021-11-14T07:20:42.582 INFO:teuthology.kill:  smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEHqiHYvIuPJcwzOB17z8hfjuK27saH6wzetdPd4aKlTPncqo2tGFwF+5gQvAYd3R+mE75x2IQyMffkqLazi9Roob9ScaHH/VGtI+ib/p/vaYvE4daLReHLLJ3KWwqzSWNFQV4SxBjK1B+sWgvhejhGXJaRwzoABsXufDJXAsfRgJxr4sGlNAqZ6vqHhr5qp9ECay14FLoyMuDIsA7EuJLESAzCvf9p5ELgNoSouPiiXWHn1pginwU0hTs0Celr2KedofWtM6aHn6rX+Sd9lXwMnsZ41VKvzjksWPf9isxd6KE0D5dVmiC9aYpvhMoTclsfF7AEkCDtuRsMe93x0UG1eD9jL+/O3aDQx+JlSbmTsfzq9Ny4hmD1W1ZHtIIWbzFDHXdOQfpVbfH/NNCh+EDO8LQmeCOWpcRu4FVsuvYP2aaT4lGPL0a+xbe9y4hS5YI/ZZQi95n2Wb3t7kryebnoM0Pd9ljuMzw6oJLYRN1QLN0QE49hBNbD1qrYitdNT0=
2021-11-14T07:20:42.607 INFO:teuthology.kill:2021-11-14 07:20:42,607.607 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-14T07:20:42.611 INFO:teuthology.kill:2021-11-14 07:20:42,610.610 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-14T07:20:42.679 INFO:teuthology.kill:2021-11-14 07:20:42,679.679 INFO:teuthology.orchestra.console:Power off smithi096
2021-11-14T07:20:42.693 INFO:teuthology.kill:2021-11-14 07:20:42,692.692 INFO:teuthology.orchestra.console:Power off smithi105
2021-11-14T07:20:51.053 INFO:teuthology.kill:2021-11-14 07:20:51,052.052 INFO:teuthology.orchestra.console:Power off for smithi096 completed
2021-11-14T07:20:51.059 INFO:teuthology.kill:2021-11-14 07:20:51,059.059 INFO:teuthology.orchestra.console:Power off for smithi105 completed
2021-11-14T07:20:51.198 INFO:teuthology.kill:2021-11-14 07:20:51,198.198 INFO:teuthology.lock.ops:unlocked: smithi096.front.sepia.ceph.com
2021-11-14T07:20:51.203 INFO:teuthology.kill:2021-11-14 07:20:51,203.203 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com
2021-11-14T07:22:51.508 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-14T07:22:51.640 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi096 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501794
2021-11-14T07:22:51.640 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi105 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-13_15:31:06-rados-wip-yuriw-master-11.12.21-distro-basic-smithi/6501793