2023-03-24T12:33:57.317 INFO:root:teuthology version: 0.0.1.dev1+ge372916
2023-03-24T12:33:57.510 INFO:teuthology.lock.ops:Start node 'smithi073.front.sepia.ceph.com' reimaging
2023-03-24T12:33:57.511 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: reset os type and version on server
2023-03-24T12:33:57.512 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server
2023-03-24T12:33:57.545 INFO:teuthology.lock.ops:Node 'smithi073.front.sepia.ceph.com' reimaging is complete
2023-03-24T12:33:57.546 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging
2023-03-24T12:33:57.546 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server
2023-03-24T12:33:57.546 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2023-03-24T12:33:57.570 INFO:teuthology.lock.ops:Node 'smithi154.front.sepia.ceph.com' reimaging is complete
2023-03-24T12:33:57.570 INFO:teuthology.lock.ops:Start node 'smithi158.front.sepia.ceph.com' reimaging
2023-03-24T12:33:57.571 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: reset os type and version on server
2023-03-24T12:33:57.571 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2023-03-24T12:33:57.596 INFO:teuthology.lock.ops:Node 'smithi158.front.sepia.ceph.com' reimaging is complete
2023-03-24T12:33:57.711 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of rhel 8.6
2023-03-24T12:33:57.723 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of rhel 8.6
2023-03-24T12:33:57.749 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of rhel 8.6
2023-03-24T12:33:57.966 INFO:teuthology.orchestra.console:Power off smithi154
2023-03-24T12:33:58.010 INFO:teuthology.orchestra.console:Power off smithi073
2023-03-24T12:33:58.027 INFO:teuthology.orchestra.console:Power off smithi158
2023-03-24T12:33:58.137 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2023-03-24T12:33:58.159 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2023-03-24T12:33:58.170 INFO:teuthology.orchestra.console:Power off for smithi158 completed
2023-03-24T12:33:58.239 INFO:teuthology.orchestra.console:Power on smithi154
2023-03-24T12:33:58.261 INFO:teuthology.orchestra.console:Power on smithi073
2023-03-24T12:33:58.280 INFO:teuthology.orchestra.console:Power on smithi158
2023-03-24T12:34:02.525 INFO:teuthology.orchestra.console:Power on for smithi073 completed
2023-03-24T12:34:02.626 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish
2023-03-24T12:34:06.628 INFO:teuthology.orchestra.console:Power on for smithi154 completed
2023-03-24T12:34:06.648 INFO:teuthology.orchestra.console:Power on for smithi158 completed
2023-03-24T12:34:06.730 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish
2023-03-24T12:34:06.749 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish
2023-03-24T12:36:51.653 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-03-24T12:36:55.749 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-03-24T12:37:00.710 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-03-24T12:37:04.805 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-03-24T12:37:08.057 ERROR:teuthology.orchestra.connection:Error authenticating with smithi158.front.sepia.ceph.com: Authentication failed.
2023-03-24T12:37:09.769 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-03-24T12:37:13.861 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-03-24T12:37:18.821 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-03-24T12:37:22.917 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-03-24T12:37:27.877 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-03-24T12:37:31.973 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-03-24T12:37:36.937 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-03-24T12:37:41.029 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-03-24T12:37:50.089 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-03-24T12:37:59.141 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-03-24T12:38:09.061 WARNING:teuthology.provision.fog:timed out
2023-03-24T12:38:42.940 WARNING:teuthology.provision.fog:timed out
2023-03-24T12:38:49.202 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-24T12:38:49.514 INFO:teuthology.provision.fog.smithi073:Node is ready
2023-03-24T12:38:49.574 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073.front.sepia.ceph.com
2023-03-24T12:38:49.636 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73 smithi073.front.sepia.ceph.com smithi073
2023-03-24T12:38:50.001 INFO:teuthology.provision.fog.smithi073:Deploy complete!
2023-03-24T12:39:05.143 WARNING:teuthology.provision.fog:timed out
2023-03-24T12:39:11.489 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-24T12:39:11.787 INFO:teuthology.provision.fog.smithi154:Node is ready
2023-03-24T12:39:11.806 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154
2023-03-24T12:39:11.866 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154
2023-03-24T12:39:12.234 INFO:teuthology.provision.fog.smithi154:Deploy complete!
2023-03-24T12:39:15.064 WARNING:teuthology.provision.fog:timed out
2023-03-24T12:39:21.393 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-24T12:39:21.700 INFO:teuthology.provision.fog.smithi158:Node is ready
2023-03-24T12:39:21.718 INFO:teuthology.orchestra.run.smithi158.stdout:smithi158.front.sepia.ceph.com
2023-03-24T12:39:21.775 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.158 smithi158.front.sepia.ceph.com smithi158
2023-03-24T12:39:22.121 INFO:teuthology.provision.fog.smithi158:Deploy complete!
2023-03-24T12:39:22.348 INFO:teuthology.lock.ops:Checking smithi073.front.sepia.ceph.com
2023-03-24T12:39:22.349 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com
2023-03-24T12:39:22.350 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-24T12:39:22.379 INFO:teuthology.lock.ops:Checking smithi158.front.sepia.ceph.com
2023-03-24T12:39:22.379 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-24T12:39:22.405 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: set os type and version on server
2023-03-24T12:39:22.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com'
2023-03-24T12:39:22.662 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64
2023-03-24T12:39:22.719 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Red Hat Enterprise Linux"
2023-03-24T12:39:22.719 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8.6 (Ootpa)"
2023-03-24T12:39:22.720 INFO:teuthology.orchestra.run.smithi073.stdout:ID="rhel"
2023-03-24T12:39:22.720 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="fedora"
2023-03-24T12:39:22.720 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8.6"
2023-03-24T12:39:22.720 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8"
2023-03-24T12:39:22.720 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-24T12:39:22.721 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31"
2023-03-24T12:39:22.721 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-24T12:39:22.721 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.redhat.com/"
2023-03-24T12:39:22.721 INFO:teuthology.orchestra.run.smithi073.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-24T12:39:22.721 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-24T12:39:22.722 INFO:teuthology.orchestra.run.smithi073.stdout:
2023-03-24T12:39:22.722 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-24T12:39:22.722 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-24T12:39:22.722 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-24T12:39:22.722 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-24T12:39:22.723 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server
2023-03-24T12:39:22.746 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server
2023-03-24T12:39:22.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com'
2023-03-24T12:39:23.001 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64
2023-03-24T12:39:23.059 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Red Hat Enterprise Linux"
2023-03-24T12:39:23.059 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8.6 (Ootpa)"
2023-03-24T12:39:23.060 INFO:teuthology.orchestra.run.smithi154.stdout:ID="rhel"
2023-03-24T12:39:23.060 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="fedora"
2023-03-24T12:39:23.060 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8.6"
2023-03-24T12:39:23.060 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8"
2023-03-24T12:39:23.060 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-24T12:39:23.060 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31"
2023-03-24T12:39:23.060 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-24T12:39:23.060 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.redhat.com/"
2023-03-24T12:39:23.061 INFO:teuthology.orchestra.run.smithi154.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-24T12:39:23.061 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-24T12:39:23.061 INFO:teuthology.orchestra.run.smithi154.stdout:
2023-03-24T12:39:23.061 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-24T12:39:23.061 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-24T12:39:23.061 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-24T12:39:23.062 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-24T12:39:23.062 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2023-03-24T12:39:23.088 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: set os type and version on server
2023-03-24T12:39:23.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-24T12:39:23.282 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64
2023-03-24T12:39:23.339 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="Red Hat Enterprise Linux"
2023-03-24T12:39:23.339 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8.6 (Ootpa)"
2023-03-24T12:39:23.340 INFO:teuthology.orchestra.run.smithi158.stdout:ID="rhel"
2023-03-24T12:39:23.340 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="fedora"
2023-03-24T12:39:23.340 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8.6"
2023-03-24T12:39:23.340 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8"
2023-03-24T12:39:23.340 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-24T12:39:23.340 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31"
2023-03-24T12:39:23.341 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-24T12:39:23.341 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://www.redhat.com/"
2023-03-24T12:39:23.341 INFO:teuthology.orchestra.run.smithi158.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-24T12:39:23.341 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-24T12:39:23.341 INFO:teuthology.orchestra.run.smithi158.stdout:
2023-03-24T12:39:23.341 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-24T12:39:23.342 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-24T12:39:23.342 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-24T12:39:23.342 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-24T12:39:23.342 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2023-03-24T12:39:23.465 INFO:teuthology.dispatcher.supervisor:Running job 7218944
2023-03-24T12:39:23.465 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_main/virtualenv/bin/teuthology -v --owner scheduled_amathuri@teuthology --archive /home/teuthworker/archive/amathuri-2023-03-24_12:28:09-fs-main-distro-default-smithi/7218944 --name amathuri-2023-03-24_12:28:09-fs-main-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} -- /home/teuthworker/archive/amathuri-2023-03-24_12:28:09-fs-main-distro-default-smithi/7218944/orig.config.yaml
2023-03-24T12:39:23.470 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/amathuri-2023-03-24_12:28:09-fs-main-distro-default-smithi/7218944
2023-03-24T12:39:23.471 INFO:teuthology.dispatcher.supervisor:Job PID: 18722
2023-03-24T12:39:23.471 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-25T00:39:47.581 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-25T00:39:47.722 INFO:teuthology.kill:Killing Pids: {18722}
2023-03-25T00:39:48.697 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0']
2023-03-25T00:39:48.699 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10']
2023-03-25T00:39:48.699 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11']
2023-03-25T00:39:48.700 INFO:teuthology.misc:Compressing logs...
2023-03-25T00:39:48.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com'
2023-03-25T00:39:48.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com'
2023-03-25T00:39:49.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2023-03-25T00:39:49.650 INFO:teuthology.misc:Compressing logs...
2023-03-25T00:39:49.939 INFO:teuthology.misc:Compressing logs...
2023-03-25T00:40:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2023-03-25T00:40:59.338 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.10.log: file size changed while zipping
2023-03-25T00:41:01.839 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.h.log: file size changed while zipping
2023-03-25T00:41:03.563 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.e.log: file size changed while zipping
2023-03-25T00:41:09.346 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2023-03-25T00:41:38.910 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2023-03-25T00:41:44.580 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2023-03-25T00:41:51.608 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2023-03-25T00:42:36.925 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.11.log: file size changed while zipping
2023-03-25T00:42:43.312 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2023-03-25T00:42:43.671 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mgr.z.log: file size changed while zipping
2023-03-25T00:42:45.228 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.k.log: file size changed while zipping
2023-03-25T00:42:52.757 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2023-03-25T00:42:54.255 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.d.log: file size changed while zipping
2023-03-25T00:43:39.580 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2023-03-25T00:43:51.363 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2023-03-25T00:43:54.699 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.8.log: file size changed while zipping
2023-03-25T00:43:56.214 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.f.log: file size changed while zipping
2023-03-25T00:43:57.681 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.c.log: file size changed while zipping
2023-03-25T00:43:59.450 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.i.log: file size changed while zipping
2023-03-25T00:44:01.165 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.b.log: file size changed while zipping
2023-03-25T00:44:16.121 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2023-03-25T00:44:17.710 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.l.log: file size changed while zipping
2023-03-25T00:44:30.441 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-osd.9.log: file size changed while zipping
2023-03-25T00:44:32.102 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mds.j.log: file size changed while zipping
2023-03-25T00:45:01.711 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/5a9d9b20-ca42-11ed-9afe-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2023-03-25T00:46:01.033 INFO:teuthology.orchestra.run.smithi073.stderr:tar: .: file changed as we read it
2023-03-25T00:47:46.708 INFO:teuthology.kill:No teuthology processes running
2023-03-25T00:47:47.578 INFO:teuthology.kill:Nuking machines: ['smithi073', 'smithi154', 'smithi158']
2023-03-25T00:47:48.226 INFO:teuthology.kill:2023-03-25 00:47:48,226.226 INFO:teuthology.nuke:targets:
2023-03-25T00:47:48.227 INFO:teuthology.kill:  smithi073.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-03-25T00:47:48.227 INFO:teuthology.kill:  smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-03-25T00:47:48.227 INFO:teuthology.kill:  smithi158.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=
2023-03-25T00:47:48.228 INFO:teuthology.kill:2023-03-25 00:47:48,226.226 INFO:teuthology.nuke:Checking targets against current locks
2023-03-25T00:47:48.283 INFO:teuthology.kill:2023-03-25 00:47:48,283.283 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-25T00:47:48.304 INFO:teuthology.kill:2023-03-25 00:47:48,303.303 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-25T00:47:48.334 INFO:teuthology.kill:2023-03-25 00:47:48,333.333 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-25T00:47:48.364 INFO:teuthology.kill:2023-03-25 00:47:48,364.364 INFO:teuthology.orchestra.console:Power off smithi073
2023-03-25T00:47:48.417 INFO:teuthology.kill:2023-03-25 00:47:48,417.417 INFO:teuthology.orchestra.console:Power off smithi154
2023-03-25T00:47:48.457 INFO:teuthology.kill:2023-03-25 00:47:48,457.457 INFO:teuthology.orchestra.console:Power off smithi158
2023-03-25T00:47:56.825 INFO:teuthology.kill:2023-03-25 00:47:56,825.825 INFO:teuthology.orchestra.console:Power off for smithi158 completed
2023-03-25T00:47:56.978 INFO:teuthology.kill:2023-03-25 00:47:56,977.977 INFO:teuthology.lock.ops:unlocked: smithi158.front.sepia.ceph.com
2023-03-25T00:48:00.867 INFO:teuthology.kill:2023-03-25 00:48:00,866.866 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2023-03-25T00:48:00.900 INFO:teuthology.kill:2023-03-25 00:48:00,900.900 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2023-03-25T00:48:01.027 INFO:teuthology.kill:2023-03-25 00:48:01,026.026 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com
2023-03-25T00:48:01.052 INFO:teuthology.kill:2023-03-25 00:48:01,051.051 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com
2023-03-25T00:50:01.308 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-25T00:50:01.461 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi073 but it was locked by another job: /home/teuthworker/archive/pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219659
2023-03-25T00:50:01.462 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi154 but it was locked by another job: /home/teuthworker/archive/pdonnell-2023-03-24_23:11:30-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219659
2023-03-25T00:50:01.462 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi158 but it was locked by another job: /home/teuthworker/archive/ivancich-2023-03-25_00:40:53-rgw:verify-main-distro-default-smithi/7219666