2023-06-16T14:48:12.702 INFO:root:teuthology version: 0.0.1.dev124+g961f4fb
2023-06-16T14:48:12.871 INFO:teuthology.lock.ops:Start node 'smithi088.front.sepia.ceph.com' reimaging
2023-06-16T14:48:12.873 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: reset os type and version on server
2023-06-16T14:48:12.873 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server
2023-06-16T14:48:12.894 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging
2023-06-16T14:48:12.895 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server
2023-06-16T14:48:12.895 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2023-06-16T14:48:13.076 INFO:teuthology.provision.fog.smithi088:Scheduling deploy of rhel 8.4
2023-06-16T14:48:13.090 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of rhel 8.4
2023-06-16T14:48:13.326 INFO:teuthology.orchestra.console:Power off smithi088
2023-06-16T14:48:13.341 INFO:teuthology.orchestra.console:Power off smithi159
2023-06-16T14:48:13.476 INFO:teuthology.orchestra.console:Power off for smithi088 completed
2023-06-16T14:48:13.487 INFO:teuthology.orchestra.console:Power off for smithi159 completed
2023-06-16T14:48:13.577 INFO:teuthology.orchestra.console:Power on smithi088
2023-06-16T14:48:13.588 INFO:teuthology.orchestra.console:Power on smithi159
2023-06-16T14:48:17.846 INFO:teuthology.orchestra.console:Power on for smithi159 completed
2023-06-16T14:48:17.947 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish
2023-06-16T14:48:21.953 INFO:teuthology.orchestra.console:Power on for smithi088 completed
2023-06-16T14:48:22.054 INFO:teuthology.provision.fog.smithi088:Waiting for deploy to finish
2023-06-16T14:51:06.900 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-06-16T14:51:15.988 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-06-16T14:51:25.072 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-06-16T14:51:25.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.88
2023-06-16T14:51:34.128 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-06-16T14:51:35.024 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.88
2023-06-16T14:51:43.184 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-06-16T14:51:44.080 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.88
2023-06-16T14:51:52.240 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-06-16T14:51:53.136 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.88
2023-06-16T14:52:01.296 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-06-16T14:52:02.192 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.88
2023-06-16T14:52:11.248 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.88
2023-06-16T14:52:20.304 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.88
2023-06-16T14:53:07.340 WARNING:teuthology.provision.fog:timed out
2023-06-16T14:53:13.654 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-16T14:53:13.930 INFO:teuthology.provision.fog.smithi159:Node is ready
2023-06-16T14:53:13.946 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159
2023-06-16T14:53:14.000 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159
2023-06-16T14:53:14.404 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Red Hat Enterprise Linux"
2023-06-16T14:53:14.404 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8.4 (Ootpa)"
2023-06-16T14:53:14.404 INFO:teuthology.orchestra.run.smithi159.stdout:ID="rhel"
2023-06-16T14:53:14.405 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="fedora"
2023-06-16T14:53:14.405 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8.4"
2023-06-16T14:53:14.405 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8"
2023-06-16T14:53:14.405 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-16T14:53:14.405 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2023-06-16T14:53:14.405 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-16T14:53:14.405 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.redhat.com/"
2023-06-16T14:53:14.406 INFO:teuthology.orchestra.run.smithi159.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-16T14:53:14.406 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-16T14:53:14.406 INFO:teuthology.orchestra.run.smithi159.stdout:
2023-06-16T14:53:14.406 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-16T14:53:14.406 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-16T14:53:14.406 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-16T14:53:14.406 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-16T14:53:14.407 INFO:teuthology.provision.fog.smithi159:Deploy complete!
2023-06-16T14:53:26.324 WARNING:teuthology.provision.fog:timed out
2023-06-16T14:53:32.601 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-16T14:53:32.870 INFO:teuthology.provision.fog.smithi088:Node is ready
2023-06-16T14:53:32.927 INFO:teuthology.orchestra.run.smithi088.stdout:smithi088.front.sepia.ceph.com
2023-06-16T14:53:32.983 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88 smithi088.front.sepia.ceph.com smithi088
2023-06-16T14:53:33.375 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="Red Hat Enterprise Linux"
2023-06-16T14:53:33.375 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="8.4 (Ootpa)"
2023-06-16T14:53:33.375 INFO:teuthology.orchestra.run.smithi088.stdout:ID="rhel"
2023-06-16T14:53:33.375 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="fedora"
2023-06-16T14:53:33.375 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="8.4"
2023-06-16T14:53:33.375 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el8"
2023-06-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31"
2023-06-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.redhat.com/"
2023-06-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi088.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi088.stdout:
2023-06-16T14:53:33.377 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-16T14:53:33.377 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-16T14:53:33.377 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-16T14:53:33.377 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-16T14:53:33.377 INFO:teuthology.provision.fog.smithi088:Deploy complete!
2023-06-16T14:53:33.528 INFO:teuthology.lock.ops:Checking smithi088.front.sepia.ceph.com
2023-06-16T14:53:33.529 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-16T14:53:33.553 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com
2023-06-16T14:53:33.553 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-16T14:53:33.579 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: set os type and version on server
2023-06-16T14:53:33.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com'
2023-06-16T14:53:33.721 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64
2023-06-16T14:53:33.775 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="Red Hat Enterprise Linux"
2023-06-16T14:53:33.776 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="8.4 (Ootpa)"
2023-06-16T14:53:33.776 INFO:teuthology.orchestra.run.smithi088.stdout:ID="rhel"
2023-06-16T14:53:33.777 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="fedora"
2023-06-16T14:53:33.777 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="8.4"
2023-06-16T14:53:33.777 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el8"
2023-06-16T14:53:33.777 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-16T14:53:33.777 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31"
2023-06-16T14:53:33.777 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-16T14:53:33.778 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.redhat.com/"
2023-06-16T14:53:33.778 INFO:teuthology.orchestra.run.smithi088.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-16T14:53:33.778 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-16T14:53:33.778 INFO:teuthology.orchestra.run.smithi088.stdout:
2023-06-16T14:53:33.778 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-16T14:53:33.778 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-16T14:53:33.778 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-16T14:53:33.779 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-16T14:53:33.779 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server
2023-06-16T14:53:33.803 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server
2023-06-16T14:53:33.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com'
2023-06-16T14:53:34.046 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64
2023-06-16T14:53:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Red Hat Enterprise Linux"
2023-06-16T14:53:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8.4 (Ootpa)"
2023-06-16T14:53:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:ID="rhel"
2023-06-16T14:53:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="fedora"
2023-06-16T14:53:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8.4"
2023-06-16T14:53:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8"
2023-06-16T14:53:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-16T14:53:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2023-06-16T14:53:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-16T14:53:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.redhat.com/"
2023-06-16T14:53:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-16T14:53:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-16T14:53:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:
2023-06-16T14:53:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-16T14:53:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-16T14:53:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-16T14:53:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-16T14:53:34.105 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2023-06-16T14:53:34.258 INFO:teuthology.dispatcher.supervisor:Running job 7306245
2023-06-16T14:53:34.259 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_961f4fb51318c373681de5844aadbc6dc0e58abc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2023-06-16_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7306245 --name teuthology-2023-06-16_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi --description upgrade:pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} -- /home/teuthworker/archive/teuthology-2023-06-16_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7306245/orig.config.yaml
2023-06-16T14:53:34.262 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2023-06-16_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7306245
2023-06-16T14:53:34.263 INFO:teuthology.dispatcher.supervisor:Job PID: 11311
2023-06-16T14:53:34.263 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-06-17T02:54:30.194 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-06-17T02:54:30.367 INFO:teuthology.kill:Killing Pids: {11311}
2023-06-17T02:54:31.319 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2023-06-17T02:54:31.320 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2023-06-17T02:54:31.320 INFO:teuthology.misc:Compressing logs...
2023-06-17T02:54:31.321 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com'
2023-06-17T02:54:31.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com'
2023-06-17T02:54:31.998 INFO:teuthology.misc:Compressing logs...
2023-06-17T02:54:32.023 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-06-17T02:54:32.037 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-06-17T02:54:32.065 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-06-17T02:54:32.069 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-06-17T02:54:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	/home/ubuntu/cephtest/archive/syslog/kern.log:	 95.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-06-17T02:54:32.136 INFO:teuthology.orchestra.run.smithi159.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	/home/ubuntu/cephtest/archive/syslog/kern.log:	 94.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-06-17T02:54:32.259 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-06-17T02:54:32.259 INFO:teuthology.orchestra.run.smithi159.stderr: 90.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-06-17T02:54:32.328 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-06-17T02:54:32.328 INFO:teuthology.orchestra.run.smithi088.stderr: 90.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-06-17T02:54:32.475 INFO:teuthology.misc:Compressing logs...
2023-06-17T02:54:32.507 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-volume.log
2023-06-17T02:54:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mds.foo.smithi088.hmnzbv.log
2023-06-17T02:54:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.c.log
2023-06-17T02:54:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.3.log
2023-06-17T02:54:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.1.log
2023-06-17T02:54:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.a.log
2023-06-17T02:54:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.0.log
2023-06-17T02:54:32.510 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mgr.y.log
2023-06-17T02:54:32.510 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.2.log
2023-06-17T02:54:32.510 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2023-06-17T02:54:32.543 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-volume.log
2023-06-17T02:54:32.543 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.7.log
2023-06-17T02:54:32.543 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mgr.x.log
2023-06-17T02:54:32.543 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.4.log
2023-06-17T02:54:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mds.foo.smithi159.xfkkxz.log
2023-06-17T02:54:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.6.log
2023-06-17T02:54:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.5.log
2023-06-17T02:54:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.b.log
2023-06-17T02:54:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2023-06-17T02:54:32.578 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-volume.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.7.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mgr.x.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.4.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mds.foo.smithi159.xfkkxz.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.6.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.5.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.b.log:	/var/log/ceph/cephadm.log:	 78.4% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mds.foo.smithi159.xfkkxz.log.gz
2023-06-17T02:54:32.649 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-volume.log:	/var/log/ceph/cephadm.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mds.foo.smithi088.hmnzbv.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.c.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.3.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.1.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.a.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.0.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mgr.y.log:	/var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.2.log:	 80.8% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mds.foo.smithi088.hmnzbv.log.gz
2023-06-17T02:54:32.697 INFO:teuthology.orchestra.run.smithi088.stderr: 93.7% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-volume.log.gz
2023-06-17T02:54:32.742 INFO:teuthology.orchestra.run.smithi159.stderr: 93.6% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-volume.log.gz
2023-06-17T02:54:33.280 INFO:teuthology.orchestra.run.smithi159.stderr: 93.5% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mgr.x.log.gz
2023-06-17T02:54:34.740 INFO:teuthology.orchestra.run.smithi088.stderr: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz
2023-06-17T02:54:34.984 INFO:teuthology.orchestra.run.smithi159.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz
2023-06-17T02:54:44.135 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2023-06-17T02:54:44.135 INFO:teuthology.orchestra.run.smithi088.stderr: 89.9% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mgr.y.log.gz
2023-06-17T02:55:02.039 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2023-06-17T02:55:02.039 INFO:teuthology.orchestra.run.smithi159.stderr: 92.5% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.b.log.gz
2023-06-17T02:55:04.541 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2023-06-17T02:55:04.542 INFO:teuthology.orchestra.run.smithi088.stderr: 92.2% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.c.log.gz
2023-06-17T02:55:49.192 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2023-06-17T02:55:49.193 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-mon.a.log.gz
2023-06-17T02:57:22.645 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2023-06-17T02:57:22.646 INFO:teuthology.orchestra.run.smithi159.stderr: 93.8% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.5.log.gz
2023-06-17T02:57:41.118 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2023-06-17T02:57:41.118 INFO:teuthology.orchestra.run.smithi159.stderr: 93.7% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.4.log.gz
2023-06-17T02:58:36.302 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2023-06-17T02:58:36.303 INFO:teuthology.orchestra.run.smithi159.stderr: 93.0% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.7.log.gz
2023-06-17T03:00:05.320 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2023-06-17T03:00:05.321 INFO:teuthology.orchestra.run.smithi159.stderr: 91.4% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.6.log.gz
2023-06-17T03:00:59.835 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2023-06-17T03:00:59.836 INFO:teuthology.orchestra.run.smithi088.stderr: 92.8% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.2.log.gz
2023-06-17T03:02:24.209 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2023-06-17T03:02:24.210 INFO:teuthology.orchestra.run.smithi088.stderr: 92.3% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.1.log.gz
2023-06-17T03:03:18.872 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2023-06-17T03:03:18.873 INFO:teuthology.orchestra.run.smithi088.stderr: 92.4% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.3.log.gz
2023-06-17T03:03:28.348 INFO:teuthology.orchestra.run.smithi088.stderr:gzip: /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2023-06-17T03:03:28.349 INFO:teuthology.orchestra.run.smithi088.stderr: 91.9% -- replaced with /var/log/ceph/0f4c792c-0c58-11ee-9b2b-001a4aab830c/ceph-osd.0.log.gz
2023-06-17T03:05:44.032 INFO:teuthology.kill:No teuthology processes running
2023-06-17T03:05:44.792 INFO:teuthology.kill:Nuking machines: ['smithi088', 'smithi159']
2023-06-17T03:05:45.399 INFO:teuthology.kill:2023-06-17 03:05:45,399.399 INFO:teuthology.nuke:targets:
2023-06-17T03:05:45.400 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_961f4fb51318c373681de5844aadbc6dc0e58abc/teuthology/dispatcher/supervisor.py", line 320, in run_with_watchdog
    kill.kill_job(
  File "/home/teuthworker/src/git.ceph.com_teuthology_961f4fb51318c373681de5844aadbc6dc0e58abc/teuthology/kill.py", line 101, in kill_job
    nuke_targets(targets, owner)
  File "/home/teuthworker/src/git.ceph.com_teuthology_961f4fb51318c373681de5844aadbc6dc0e58abc/teuthology/kill.py", line 276, in nuke_targets
    sys.stdout.flush()
OSError: [Errno 5] Input/output error
2023-06-17T03:07:45.590 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15