2023-06-13T22:58:29.104 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd
2023-06-13T22:58:29.257 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging
2023-06-13T22:58:29.259 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server
2023-06-13T22:58:29.259 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2023-06-13T22:58:29.450 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of rhel 8.6
2023-06-13T22:58:29.684 INFO:teuthology.orchestra.console:Power off smithi098
2023-06-13T22:58:38.054 INFO:teuthology.orchestra.console:Power off for smithi098 completed
2023-06-13T22:58:38.155 INFO:teuthology.orchestra.console:Power on smithi098
2023-06-13T22:58:46.601 INFO:teuthology.orchestra.console:Power on for smithi098 completed
2023-06-13T22:58:46.702 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish
2023-06-13T23:01:51.092 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-06-13T23:02:00.176 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-06-13T23:02:09.232 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-06-13T23:02:18.288 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-06-13T23:03:24.352 WARNING:teuthology.provision.fog:timed out
2023-06-13T23:03:30.697 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-13T23:03:30.990 INFO:teuthology.provision.fog.smithi098:Node is ready
2023-06-13T23:03:31.008 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com
2023-06-13T23:03:31.066 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098
2023-06-13T23:03:31.474 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux"
2023-06-13T23:03:31.474 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)"
2023-06-13T23:03:31.474 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel"
2023-06-13T23:03:31.474 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora"
2023-06-13T23:03:31.474 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6"
2023-06-13T23:03:31.475 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8"
2023-06-13T23:03:31.475 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-06-13T23:03:31.475 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2023-06-13T23:03:31.475 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-06-13T23:03:31.475 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/"
2023-06-13T23:03:31.475 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-13T23:03:31.475 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-13T23:03:31.475 INFO:teuthology.orchestra.run.smithi098.stdout:
2023-06-13T23:03:31.476 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-13T23:03:31.476 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-06-13T23:03:31.476 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-13T23:03:31.476 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-06-13T23:03:31.477 INFO:teuthology.provision.fog.smithi098:Deploy complete!
2023-06-13T23:03:31.607 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com
2023-06-13T23:03:31.609 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-13T23:03:31.633 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server
2023-06-13T23:03:31.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-06-13T23:03:31.818 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64
2023-06-13T23:03:31.875 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux"
2023-06-13T23:03:31.875 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)"
2023-06-13T23:03:31.875 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel"
2023-06-13T23:03:31.875 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora"
2023-06-13T23:03:31.876 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6"
2023-06-13T23:03:31.876 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8"
2023-06-13T23:03:31.876 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-06-13T23:03:31.876 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2023-06-13T23:03:31.876 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-06-13T23:03:31.877 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/"
2023-06-13T23:03:31.877 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-13T23:03:31.877 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-13T23:03:31.877 INFO:teuthology.orchestra.run.smithi098.stdout:
2023-06-13T23:03:31.877 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-13T23:03:31.877 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-06-13T23:03:31.877 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-13T23:03:31.878 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-06-13T23:03:31.878 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2023-06-13T23:03:31.996 INFO:teuthology.dispatcher.supervisor:Running job 7302794
2023-06-13T23:03:31.997 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-13_18:33:48-rados-wip-yuri10-testing-2023-06-02-1406-distro-default-smithi/7302794 --name yuriw-2023-06-13_18:33:48-rados-wip-yuri10-testing-2023-06-02-1406-distro-default-smithi --description rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} -- /home/teuthworker/archive/yuriw-2023-06-13_18:33:48-rados-wip-yuri10-testing-2023-06-02-1406-distro-default-smithi/7302794/orig.config.yaml
2023-06-13T23:03:32.000 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-06-13_18:33:48-rados-wip-yuri10-testing-2023-06-02-1406-distro-default-smithi/7302794
2023-06-13T23:03:32.001 INFO:teuthology.dispatcher.supervisor:Job PID: 10935
2023-06-13T23:03:32.001 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-06-14T11:04:46.909 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-06-14T11:04:51.490 INFO:teuthology.kill:Killing Pids: {10935}
2023-06-14T11:04:55.001 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.a', 'mds.a', 'mgr.x', 'osd.0', 'osd.1', 'client.0']
2023-06-14T11:04:55.108 INFO:teuthology.misc:Compressing logs...
2023-06-14T11:04:55.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-06-14T11:05:03.332 INFO:teuthology.misc:Compressing logs...
2023-06-14T11:05:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-06-14T11:05:03.405 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-06-14T11:05:03.405 INFO:teuthology.orchestra.run.smithi098.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	 80.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-06-14T11:05:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	 88.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-06-14T11:05:03.586 INFO:teuthology.misc:Compressing logs...
2023-06-14T11:05:03.618 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95355.log
2023-06-14T11:05:03.618 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95961.log
2023-06-14T11:05:03.620 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95813.log
2023-06-14T11:05:03.620 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95355.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95355.log.gz
2023-06-14T11:05:03.620 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95961.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94253.log
2023-06-14T11:05:03.620 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95961.log.gz
2023-06-14T11:05:03.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95813.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97297.log
2023-06-14T11:05:03.621 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95813.log.gz
2023-06-14T11:05:03.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.94253.log:	gzip -5 --verbose -- /var/log/ceph/ceph.log
2023-06-14T11:05:03.621 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.94253.log.gz
2023-06-14T11:05:03.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97297.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93946.log
2023-06-14T11:05:03.621 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97297.log.gz
2023-06-14T11:05:03.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95861.log
2023-06-14T11:05:03.623 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.93946.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97503.log
2023-06-14T11:05:03.623 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.93946.log.gz
2023-06-14T11:05:03.623 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95861.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95401.log
2023-06-14T11:05:03.623 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95861.log.gz
2023-06-14T11:05:03.623 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97503.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97503.log.gz
2023-06-14T11:05:03.623 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log
2023-06-14T11:05:03.623 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95401.log:	gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log
2023-06-14T11:05:03.624 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95401.log.gz
2023-06-14T11:05:03.627 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ops-log-ceph-client.0.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96537.log
2023-06-14T11:05:03.628 INFO:teuthology.orchestra.run.smithi098.stderr: 34.7% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz
2023-06-14T11:05:03.628 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.0.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96883.log
2023-06-14T11:05:03.628 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96537.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96537.log.gz
2023-06-14T11:05:03.628 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97593.log
2023-06-14T11:05:03.628 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96883.log:	gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log
2023-06-14T11:05:03.628 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96883.log.gz
2023-06-14T11:05:03.629 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97593.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96837.log
2023-06-14T11:05:03.629 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97593.log.gz
2023-06-14T11:05:03.629 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mds.a.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93841.log
2023-06-14T11:05:03.629 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96837.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96377.log
2023-06-14T11:05:03.629 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96837.log.gz
2023-06-14T11:05:03.629 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.93841.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.93841.log.gz
2023-06-14T11:05:03.629 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96007.log
2023-06-14T11:05:03.629 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97067.log
2023-06-14T11:05:03.630 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96377.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96377.log.gz
2023-06-14T11:05:03.630 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96007.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96101.log
2023-06-14T11:05:03.630 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96007.log.gz
2023-06-14T11:05:03.630 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97067.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97067.log.gz
2023-06-14T11:05:03.630 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95469.log
2023-06-14T11:05:03.630 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97251.log
2023-06-14T11:05:03.630 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96101.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96101.log.gz
2023-06-14T11:05:03.630 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95469.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96607.log
2023-06-14T11:05:03.636 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95469.log.gz
2023-06-14T11:05:03.636 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz
2023-06-14T11:05:03.636 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97251.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97251.log.gz
2023-06-14T11:05:03.636 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96331.log
2023-06-14T11:05:03.636 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95627.log
2023-06-14T11:05:03.636 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96607.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96607.log.gz
2023-06-14T11:05:03.636 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96331.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95581.log
2023-06-14T11:05:03.637 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96331.log.gz
2023-06-14T11:05:03.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95627.log:	gzip -5 --verbose --   0.0%/var/log/ceph/ceph-client.admin.96445.log
2023-06-14T11:05:03.637 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95627.log.gz
2023-06-14T11:05:03.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95581.log:	gzip -5 --verbose -- /var/log/ceph/ceph.audit.log
2023-06-14T11:05:03.637 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95581.log.gz
2023-06-14T11:05:03.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96445.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96147.log
2023-06-14T11:05:03.637 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96445.log.gz
2023-06-14T11:05:03.638 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.audit.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96791.log
2023-06-14T11:05:03.638 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96147.log:	 85.5% -- replaced with /var/log/ceph/ceph.audit.log.gz
2023-06-14T11:05:03.638 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96147.log.gz
2023-06-14T11:05:03.638 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100473.log
2023-06-14T11:05:03.638 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96791.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95915.log
2023-06-14T11:05:03.638 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96791.log.gz
2023-06-14T11:05:03.638 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.100473.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.100473.log.gz
2023-06-14T11:05:03.638 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97343.log
2023-06-14T11:05:03.639 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95915.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95535.log
2023-06-14T11:05:03.639 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95915.log.gz
2023-06-14T11:05:03.639 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97343.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95292.log
2023-06-14T11:05:03.639 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97343.log.gz
2023-06-14T11:05:03.639 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95535.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96193.log
2023-06-14T11:05:03.639 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95535.log.gz
2023-06-14T11:05:03.639 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95292.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97113.log
2023-06-14T11:05:03.640 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95292.log.gz
2023-06-14T11:05:03.640 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96193.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96538.log
2023-06-14T11:05:03.640 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96193.log.gz
2023-06-14T11:05:03.640 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97113.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97113.log.gz
2023-06-14T11:05:03.640 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log
2023-06-14T11:05:03.640 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94198.log
2023-06-14T11:05:03.640 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96538.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96538.log.gz
2023-06-14T11:05:03.640 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/rgw.ceph.client.0.log:	gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log
2023-06-14T11:05:03.641 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.94198.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96745.log
2023-06-14T11:05:03.641 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.94198.log.gz
2023-06-14T11:05:03.641 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.1.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97389.log
2023-06-14T11:05:03.641 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96745.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96745.log.gz
2023-06-14T11:05:03.641 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log
2023-06-14T11:05:03.641 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97389.log:	 87.7% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz
2023-06-14T11:05:03.641 INFO:teuthology.orchestra.run.smithi098.stderr: 48.1% -- replaced with /var/log/ceph/ceph-client.admin.97389.log.gz
2023-06-14T11:05:03.641 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96630.log
2023-06-14T11:05:03.642 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 /var/log/ceph/ceph-mgr.x.log:	--verbose -- /var/log/ceph/ceph-client.admin.95719.log
2023-06-14T11:05:03.642 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96630.log:	gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.91363.log
2023-06-14T11:05:03.642 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96630.log.gz
2023-06-14T11:05:03.642 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95719.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95765.log
2023-06-14T11:05:03.642 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95719.log.gz
2023-06-14T11:05:03.642 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.tmp-client.admin.91363.log:	gzip   0.0%-5 --verbose --  -- replaced with /var/log/ceph/ceph.tmp-client.admin.91363.log.gz/var/log/ceph/ceph-client.admin.96285.log
2023-06-14T11:05:03.642 INFO:teuthology.orchestra.run.smithi098.stderr:
2023-06-14T11:05:03.643 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95765.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96446.log
2023-06-14T11:05:03.643 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95765.log.gz
2023-06-14T11:05:03.643 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96285.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96285.log.gz
2023-06-14T11:05:03.643 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97547.log
2023-06-14T11:05:03.643 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96446.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97457.log
2023-06-14T11:05:03.643 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96446.log.gz
2023-06-14T11:05:03.646 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97547.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97547.log.gz
2023-06-14T11:05:03.646 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96929.log
2023-06-14T11:05:03.646 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97457.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97457.log.gz
2023-06-14T11:05:03.646 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96929.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96929.log.gz
2023-06-14T11:05:03.646 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96699.log
2023-06-14T11:05:03.646 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96975.log
2023-06-14T11:05:03.647 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96699.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96699.log.gz
2023-06-14T11:05:03.647 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96975.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96975.log.gz
2023-06-14T11:05:03.647 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97159.log
2023-06-14T11:05:03.647 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97021.log
2023-06-14T11:05:03.647 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97159.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97159.log.gz
2023-06-14T11:05:03.647 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97021.log:	  0.0%gzip -5  -- replaced with /var/log/ceph/ceph-client.admin.97021.log.gz--verbose
2023-06-14T11:05:03.647 INFO:teuthology.orchestra.run.smithi098.stderr:-- /var/log/ceph/ceph-client.admin.96055.log
2023-06-14T11:05:03.648 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94060.log
2023-06-14T11:05:03.648 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96055.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96055.log.gz
2023-06-14T11:05:03.648 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96239.log
2023-06-14T11:05:03.648 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97205.log
2023-06-14T11:05:03.648 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.94060.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.94060.log.gz
2023-06-14T11:05:03.648 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.96239.log:	gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log
2023-06-14T11:05:03.648 INFO:teuthology.orchestra.run.smithi098.stderr:  0.0% -- replaced with /var/log/ceph/ceph-client.admin.96239.log.gz
2023-06-14T11:05:03.648 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.97205.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.97205.log.gz
2023-06-14T11:05:03.649 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mon.a.log:	gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94107.log
2023-06-14T11:05:03.649 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95673.log
2023-06-14T11:05:03.649 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.94107.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.94107.log.gz
2023-06-14T11:05:03.649 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.95673.log:	  0.0% -- replaced with /var/log/ceph/ceph-client.admin.95673.log.gz
2023-06-14T11:05:03.662 INFO:teuthology.orchestra.run.smithi098.stderr: 90.8% -- replaced with /var/log/ceph/ceph.log.gz
2023-06-14T11:05:05.973 INFO:teuthology.orchestra.run.smithi098.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz
2023-06-14T11:05:14.537 INFO:teuthology.orchestra.run.smithi098.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz
2023-06-14T11:06:16.801 INFO:teuthology.orchestra.run.smithi098.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz
2023-06-14T11:06:17.448 INFO:teuthology.orchestra.run.smithi098.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz
2023-06-14T11:06:35.551 INFO:teuthology.kill:No teuthology processes running
2023-06-14T11:06:37.291 INFO:teuthology.kill:Nuking machines: ['smithi098']
2023-06-14T11:06:37.994 INFO:teuthology.kill:2023-06-14 11:06:37,993.993 INFO:teuthology.nuke:targets:
2023-06-14T11:06:38.035 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/dispatcher/supervisor.py", line 320, in run_with_watchdog
    kill.kill_job(
  File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/kill.py", line 101, in kill_job
    nuke_targets(targets, owner)
  File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/kill.py", line 276, in nuke_targets
    sys.stdout.flush()
OSError: [Errno 5] Input/output error
2023-06-14T11:08:38.348 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-06-14T11:08:38.515 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi098 but it was locked by another job: /home/teuthworker/archive/nmordech-2023-06-14_10:39:42-rados:monthrash-main-distro-default-smithi/7304010