2023-01-30T13:31:38.946 INFO:root:teuthology version: 0.0.1.dev33+gbf44874
2023-01-30T13:31:39.169 INFO:teuthology.lock.ops:Start node 'smithi046.front.sepia.ceph.com' reimaging
2023-01-30T13:31:39.170 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: reset os type and version on server
2023-01-30T13:31:39.170 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server
2023-01-30T13:31:39.188 INFO:teuthology.lock.ops:Node 'smithi046.front.sepia.ceph.com' reimaging is complete
2023-01-30T13:31:39.188 INFO:teuthology.lock.ops:Start node 'smithi106.front.sepia.ceph.com' reimaging
2023-01-30T13:31:39.189 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: reset os type and version on server
2023-01-30T13:31:39.189 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2023-01-30T13:31:39.206 INFO:teuthology.lock.ops:Node 'smithi106.front.sepia.ceph.com' reimaging is complete
2023-01-30T13:31:39.206 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging
2023-01-30T13:31:39.207 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server
2023-01-30T13:31:39.207 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2023-01-30T13:31:39.231 INFO:teuthology.lock.ops:Node 'smithi167.front.sepia.ceph.com' reimaging is complete
2023-01-30T13:31:39.341 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of rhel 8.6
2023-01-30T13:31:39.382 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of rhel 8.6
2023-01-30T13:31:39.385 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of rhel 8.6
2023-01-30T13:31:39.628 INFO:teuthology.orchestra.console:Power off smithi046
2023-01-30T13:31:39.669 INFO:teuthology.orchestra.console:Power off smithi106
2023-01-30T13:31:39.688 INFO:teuthology.orchestra.console:Power off smithi167
2023-01-30T13:31:39.798 INFO:teuthology.orchestra.console:Power off for smithi046 completed
2023-01-30T13:31:39.833 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2023-01-30T13:31:39.850 INFO:teuthology.orchestra.console:Power off for smithi167 completed
2023-01-30T13:31:39.900 INFO:teuthology.orchestra.console:Power on smithi046
2023-01-30T13:31:39.935 INFO:teuthology.orchestra.console:Power on smithi106
2023-01-30T13:31:39.951 INFO:teuthology.orchestra.console:Power on smithi167
2023-01-30T13:31:44.173 INFO:teuthology.orchestra.console:Power on for smithi046 completed
2023-01-30T13:31:44.196 INFO:teuthology.orchestra.console:Power on for smithi106 completed
2023-01-30T13:31:44.274 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish
2023-01-30T13:31:44.298 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish
2023-01-30T13:31:48.361 INFO:teuthology.orchestra.console:Power on for smithi167 completed
2023-01-30T13:31:48.463 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish
2023-01-30T13:34:48.424 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106
2023-01-30T13:34:52.520 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-30T13:34:57.512 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106
2023-01-30T13:35:01.576 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-30T13:35:06.568 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106
2023-01-30T13:35:10.632 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-30T13:35:15.099 ERROR:teuthology.orchestra.connection:Error authenticating with smithi046.front.sepia.ceph.com: Authentication timeout.
2023-01-30T13:35:15.624 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106
2023-01-30T13:35:19.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-30T13:35:28.744 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-30T13:36:06.248 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-01-30T13:36:06.727 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-30T13:36:07.080 INFO:teuthology.provision.fog.smithi167:Node is ready
2023-01-30T13:36:07.095 INFO:teuthology.orchestra.run.smithi167.stdout:smithi022
2023-01-30T13:36:07.402 INFO:teuthology.provision.fog.smithi167:Deploy complete!
2023-01-30T13:36:16.116 WARNING:teuthology.provision.fog:timed out
2023-01-30T13:36:21.633 WARNING:teuthology.provision.fog:timed out
2023-01-30T13:36:22.352 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-30T13:36:22.645 INFO:teuthology.provision.fog.smithi046:Node is ready
2023-01-30T13:36:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com
2023-01-30T13:36:22.723 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046
2023-01-30T13:36:23.091 INFO:teuthology.provision.fog.smithi046:Deploy complete!
2023-01-30T13:36:27.894 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-30T13:36:28.222 INFO:teuthology.provision.fog.smithi106:Node is ready
2023-01-30T13:36:28.316 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com
2023-01-30T13:36:28.374 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106
2023-01-30T13:36:28.722 INFO:teuthology.provision.fog.smithi106:Deploy complete!
2023-01-30T13:36:28.912 INFO:teuthology.lock.ops:Checking smithi046.front.sepia.ceph.com
2023-01-30T13:36:28.913 INFO:teuthology.lock.ops:New key found. Updating...
2023-01-30T13:36:28.931 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com
2023-01-30T13:36:28.931 INFO:teuthology.lock.ops:New key found. Updating...
2023-01-30T13:36:28.946 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com
2023-01-30T13:36:28.946 INFO:teuthology.lock.ops:New key found. Updating...
2023-01-30T13:36:28.961 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: set os type and version on server
2023-01-30T13:36:28.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com'
2023-01-30T13:36:29.160 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64
2023-01-30T13:36:29.218 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Red Hat Enterprise Linux"
2023-01-30T13:36:29.218 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="8.6 (Ootpa)"
2023-01-30T13:36:29.219 INFO:teuthology.orchestra.run.smithi046.stdout:ID="rhel"
2023-01-30T13:36:29.219 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="fedora"
2023-01-30T13:36:29.219 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="8.6"
2023-01-30T13:36:29.219 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el8"
2023-01-30T13:36:29.219 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-01-30T13:36:29.220 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31"
2023-01-30T13:36:29.220 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-01-30T13:36:29.220 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.redhat.com/"
2023-01-30T13:36:29.220 INFO:teuthology.orchestra.run.smithi046.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-01-30T13:36:29.221 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-01-30T13:36:29.221 INFO:teuthology.orchestra.run.smithi046.stdout:
2023-01-30T13:36:29.221 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-01-30T13:36:29.221 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-01-30T13:36:29.221 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-01-30T13:36:29.222 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-01-30T13:36:29.222 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server
2023-01-30T13:36:29.238 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server
2023-01-30T13:36:29.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com'
2023-01-30T13:36:29.428 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64
2023-01-30T13:36:29.486 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Red Hat Enterprise Linux"
2023-01-30T13:36:29.486 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8.6 (Ootpa)"
2023-01-30T13:36:29.486 INFO:teuthology.orchestra.run.smithi106.stdout:ID="rhel"
2023-01-30T13:36:29.487 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="fedora"
2023-01-30T13:36:29.487 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8.6"
2023-01-30T13:36:29.487 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8"
2023-01-30T13:36:29.487 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-01-30T13:36:29.487 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31"
2023-01-30T13:36:29.488 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-01-30T13:36:29.488 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.redhat.com/"
2023-01-30T13:36:29.488 INFO:teuthology.orchestra.run.smithi106.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-01-30T13:36:29.488 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-01-30T13:36:29.488 INFO:teuthology.orchestra.run.smithi106.stdout:
2023-01-30T13:36:29.489 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-01-30T13:36:29.489 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-01-30T13:36:29.489 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-01-30T13:36:29.489 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-01-30T13:36:29.490 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2023-01-30T13:36:29.528 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server
2023-01-30T13:36:29.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2023-01-30T13:36:29.783 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64
2023-01-30T13:36:29.840 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux"
2023-01-30T13:36:29.840 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)"
2023-01-30T13:36:29.840 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel"
2023-01-30T13:36:29.841 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora"
2023-01-30T13:36:29.841 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6"
2023-01-30T13:36:29.841 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8"
2023-01-30T13:36:29.841 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-01-30T13:36:29.841 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2023-01-30T13:36:29.842 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-01-30T13:36:29.842 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/"
2023-01-30T13:36:29.842 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-01-30T13:36:29.842 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-01-30T13:36:29.842 INFO:teuthology.orchestra.run.smithi167.stdout:
2023-01-30T13:36:29.843 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-01-30T13:36:29.843 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-01-30T13:36:29.843 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-01-30T13:36:29.843 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-01-30T13:36:29.844 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2023-01-30T13:36:29.937 INFO:teuthology.dispatcher.supervisor:Running job 7143870
2023-01-30T13:36:29.938 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_bf4487485b740895877d58f6fd36a4e39d847c23/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-01-30_12:35:25-fs-wip-vshankar-testing-20230125.055346-testing-default-smithi/7143870 --name vshankar-2023-01-30_12:35:25-fs-wip-vshankar-testing-20230125.055346-testing-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} -- /home/teuthworker/archive/vshankar-2023-01-30_12:35:25-fs-wip-vshankar-testing-20230125.055346-testing-default-smithi/7143870/orig.config.yaml
2023-01-30T13:36:29.941 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-01-30_12:35:25-fs-wip-vshankar-testing-20230125.055346-testing-default-smithi/7143870
2023-01-30T13:36:29.941 INFO:teuthology.dispatcher.supervisor:Job PID: 11560
2023-01-30T13:36:29.942 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-01-31T01:37:20.421 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-01-31T01:37:20.633 INFO:teuthology.kill:Killing Pids: {11560}
2023-01-31T01:37:24.479 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0']
2023-01-31T01:37:24.504 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10']
2023-01-31T01:37:24.504 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11']
2023-01-31T01:37:24.504 INFO:teuthology.misc:Compressing logs...
2023-01-31T01:37:24.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com'
2023-01-31T01:37:24.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com'
2023-01-31T01:37:24.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2023-01-31T01:37:25.419 INFO:teuthology.misc:Compressing logs...
2023-01-31T01:37:26.142 INFO:teuthology.misc:Compressing logs...
2023-01-31T01:38:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2023-01-31T01:38:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2023-01-31T01:39:12.757 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-osd.11.log: file size changed while zipping
2023-01-31T01:39:17.763 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-mgr.z.log: file size changed while zipping
2023-01-31T01:39:50.276 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-osd.8.log: file size changed while zipping
2023-01-31T01:39:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-mds.f.log: file size changed while zipping
2023-01-31T01:39:55.251 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-mds.c.log: file size changed while zipping
2023-01-31T01:39:58.146 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-mds.i.log: file size changed while zipping
2023-01-31T01:40:19.399 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2023-01-31T01:40:20.884 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-mds.l.log: file size changed while zipping
2023-01-31T04:18:55.654 INFO:teuthology.orchestra.run.smithi046.stderr:
2023-01-31T04:18:55.670 INFO:teuthology.orchestra.run.smithi046.stderr:gzip: /var/log/ceph/daab4716-a0a5-11ed-9e56-001a4aab830c/ceph-mon.a.log.gz: No space left on device
2023-01-31T04:18:58.883 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_bf4487485b740895877d58f6fd36a4e39d847c23/teuthology/dispatcher/supervisor.py", line 289, in run_with_watchdog
    transfer_archives(job_info['name'], job_info['job_id'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_bf4487485b740895877d58f6fd36a4e39d847c23/teuthology/dispatcher/supervisor.py", line 348, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_teuthology_bf4487485b740895877d58f6fd36a4e39d847c23/teuthology/misc.py", line 1377, in compress_logs
    run.wait(
  File "/home/teuthworker/src/git.ceph.com_teuthology_bf4487485b740895877d58f6fd36a4e39d847c23/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_teuthology_bf4487485b740895877d58f6fd36a4e39d847c23/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_teuthology_bf4487485b740895877d58f6fd36a4e39d847c23/teuthology/orchestra/run.py", line 181, in _raise_for_status
    raise CommandFailedError(
teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 123: 'sudo find /var/log/ceph -name *.log -print0 | sudo xargs -0 --no-run-if-empty -- gzip --'
2023-01-31T04:18:58.998 INFO:teuthology.kill:No teuthology processes running
2023-01-31T04:19:03.396 INFO:teuthology.kill:Nuking machines: ['smithi046', 'smithi106', 'smithi167']
2023-01-31T04:19:04.121 INFO:teuthology.kill:2023-01-31 04:19:04,121.121 INFO:teuthology.nuke:targets:
2023-01-31T04:19:04.122 INFO:teuthology.kill:  smithi046.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-01-31T04:19:04.122 INFO:teuthology.kill:  smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-01-31T04:19:04.122 INFO:teuthology.kill:  smithi167.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-01-31T04:19:04.123 INFO:teuthology.kill:2023-01-31 04:19:04,121.121 INFO:teuthology.nuke:Checking targets against current locks
2023-01-31T04:19:04.189 INFO:teuthology.kill:2023-01-31 04:19:04,189.189 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-31T04:19:04.212 INFO:teuthology.kill:2023-01-31 04:19:04,212.212 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-31T04:19:04.238 INFO:teuthology.kill:2023-01-31 04:19:04,237.237 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-31T04:19:04.268 INFO:teuthology.kill:2023-01-31 04:19:04,268.268 INFO:teuthology.orchestra.console:Power off smithi046
2023-01-31T04:19:04.303 INFO:teuthology.kill:2023-01-31 04:19:04,303.303 INFO:teuthology.orchestra.console:Power off smithi106
2023-01-31T04:19:04.332 INFO:teuthology.kill:2023-01-31 04:19:04,332.332 INFO:teuthology.orchestra.console:Power off smithi167
2023-01-31T04:19:12.633 INFO:teuthology.kill:2023-01-31 04:19:12,633.633 INFO:teuthology.orchestra.console:Power off for smithi046 completed
2023-01-31T04:19:12.660 INFO:teuthology.kill:2023-01-31 04:19:12,660.660 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2023-01-31T04:19:12.690 INFO:teuthology.kill:2023-01-31 04:19:12,690.690 INFO:teuthology.orchestra.console:Power off for smithi167 completed
2023-01-31T04:19:12.787 INFO:teuthology.kill:2023-01-31 04:19:12,787.787 INFO:teuthology.lock.ops:unlocked: smithi046.front.sepia.ceph.com
2023-01-31T04:19:12.812 INFO:teuthology.kill:2023-01-31 04:19:12,812.812 INFO:teuthology.lock.ops:unlocked: smithi106.front.sepia.ceph.com
2023-01-31T04:19:12.836 INFO:teuthology.kill:2023-01-31 04:19:12,836.836 INFO:teuthology.lock.ops:unlocked: smithi167.front.sepia.ceph.com
2023-01-31T04:21:13.068 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-01-31T04:21:13.219 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi106 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-30_23:06:08-rbd-wip-yuri6-testing-2023-01-26-0941-distro-default-smithi/7144169
2023-01-31T04:21:13.219 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi167 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-30_23:06:08-rbd-wip-yuri6-testing-2023-01-26-0941-distro-default-smithi/7144169