2023-06-14T07:30:18.227 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd
2023-06-14T07:30:18.394 INFO:teuthology.lock.ops:Start node 'smithi151.front.sepia.ceph.com' reimaging
2023-06-14T07:30:18.396 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: reset os type and version on server
2023-06-14T07:30:18.396 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server
2023-06-14T07:30:18.423 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging
2023-06-14T07:30:18.423 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server
2023-06-14T07:30:18.424 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2023-06-14T07:30:18.601 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of centos 8.stream
2023-06-14T07:30:18.606 INFO:teuthology.provision.fog.smithi151:Scheduling deploy of centos 8.stream
2023-06-14T07:30:18.816 INFO:teuthology.orchestra.console:Power off smithi151
2023-06-14T07:30:18.832 INFO:teuthology.orchestra.console:Power off smithi191
2023-06-14T07:30:27.195 INFO:teuthology.orchestra.console:Power off for smithi151 completed
2023-06-14T07:30:27.297 INFO:teuthology.orchestra.console:Power on smithi151
2023-06-14T07:30:31.316 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2023-06-14T07:30:31.418 INFO:teuthology.orchestra.console:Power on smithi191
2023-06-14T07:30:35.666 INFO:teuthology.orchestra.console:Power on for smithi151 completed
2023-06-14T07:30:35.673 INFO:teuthology.orchestra.console:Power on for smithi191 completed
2023-06-14T07:30:35.768 INFO:teuthology.provision.fog.smithi151:Waiting for deploy to finish
2023-06-14T07:30:35.775 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish
2023-06-14T07:33:40.084 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2023-06-14T07:33:40.086 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-14T07:33:49.168 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2023-06-14T07:33:49.169 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-14T07:33:58.224 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2023-06-14T07:33:58.225 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-14T07:34:07.284 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2023-06-14T07:34:07.285 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-14T07:34:16.372 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2023-06-14T07:34:16.373 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-14T07:34:25.460 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-14T07:35:22.424 WARNING:teuthology.provision.fog:timed out
2023-06-14T07:35:28.625 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-14T07:35:28.868 INFO:teuthology.provision.fog.smithi191:Node is ready
2023-06-14T07:35:28.887 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com
2023-06-14T07:35:28.945 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191
2023-06-14T07:35:29.335 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream"
2023-06-14T07:35:29.335 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8"
2023-06-14T07:35:29.336 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos"
2023-06-14T07:35:29.336 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora"
2023-06-14T07:35:29.336 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8"
2023-06-14T07:35:29.336 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8"
2023-06-14T07:35:29.336 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-14T07:35:29.336 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31"
2023-06-14T07:35:29.337 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-14T07:35:29.337 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/"
2023-06-14T07:35:29.337 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-14T07:35:29.337 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-14T07:35:29.337 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-14T07:35:29.338 INFO:teuthology.provision.fog.smithi191:Deploy complete!
2023-06-14T07:35:31.468 WARNING:teuthology.provision.fog:timed out
2023-06-14T07:35:37.712 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-14T07:35:37.946 INFO:teuthology.provision.fog.smithi151:Node is ready
2023-06-14T07:35:37.966 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151.front.sepia.ceph.com
2023-06-14T07:35:38.026 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151 smithi151.front.sepia.ceph.com smithi151
2023-06-14T07:35:38.425 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream"
2023-06-14T07:35:38.426 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8"
2023-06-14T07:35:38.426 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos"
2023-06-14T07:35:38.426 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora"
2023-06-14T07:35:38.426 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8"
2023-06-14T07:35:38.426 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8"
2023-06-14T07:35:38.427 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-14T07:35:38.427 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31"
2023-06-14T07:35:38.427 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-14T07:35:38.427 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/"
2023-06-14T07:35:38.427 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-14T07:35:38.427 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-14T07:35:38.428 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-14T07:35:38.428 INFO:teuthology.provision.fog.smithi151:Deploy complete!
2023-06-14T07:35:38.583 INFO:teuthology.lock.ops:Checking smithi151.front.sepia.ceph.com
2023-06-14T07:35:38.584 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-14T07:35:38.634 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com
2023-06-14T07:35:38.634 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-14T07:35:38.660 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: set os type and version on server
2023-06-14T07:35:38.660 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com'
2023-06-14T07:35:38.861 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64
2023-06-14T07:35:38.919 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream"
2023-06-14T07:35:38.919 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8"
2023-06-14T07:35:38.919 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos"
2023-06-14T07:35:38.920 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora"
2023-06-14T07:35:38.920 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8"
2023-06-14T07:35:38.920 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8"
2023-06-14T07:35:38.920 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-14T07:35:38.920 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31"
2023-06-14T07:35:38.920 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-14T07:35:38.921 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/"
2023-06-14T07:35:38.921 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-14T07:35:38.921 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-14T07:35:38.921 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-14T07:35:38.922 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server
2023-06-14T07:35:38.950 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server
2023-06-14T07:35:38.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com'
2023-06-14T07:35:39.158 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64
2023-06-14T07:35:39.219 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream"
2023-06-14T07:35:39.219 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8"
2023-06-14T07:35:39.219 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos"
2023-06-14T07:35:39.219 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora"
2023-06-14T07:35:39.219 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8"
2023-06-14T07:35:39.219 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8"
2023-06-14T07:35:39.220 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-14T07:35:39.220 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31"
2023-06-14T07:35:39.220 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-14T07:35:39.220 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/"
2023-06-14T07:35:39.220 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-14T07:35:39.220 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-14T07:35:39.220 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-14T07:35:39.221 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2023-06-14T07:35:39.349 INFO:teuthology.dispatcher.supervisor:Running job 7303095
2023-06-14T07:35:39.350 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2023-06-13_18:58:18-orch:cephadm-wip-adk4-testing-2023-06-13-1048-distro-default-smithi/7303095 --name adking-2023-06-13_18:58:18-orch:cephadm-wip-adk4-testing-2023-06-13-1048-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/adking-2023-06-13_18:58:18-orch:cephadm-wip-adk4-testing-2023-06-13-1048-distro-default-smithi/7303095/orig.config.yaml
2023-06-14T07:35:39.353 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2023-06-13_18:58:18-orch:cephadm-wip-adk4-testing-2023-06-13-1048-distro-default-smithi/7303095
2023-06-14T07:35:39.354 INFO:teuthology.dispatcher.supervisor:Job PID: 2291
2023-06-14T07:35:39.354 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-06-14T19:36:41.717 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-06-14T19:36:41.890 INFO:teuthology.kill:Killing Pids: {2291}
2023-06-14T19:36:42.742 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2023-06-14T19:36:42.743 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2023-06-14T19:36:42.743 INFO:teuthology.misc:Compressing logs...
2023-06-14T19:36:42.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com'
2023-06-14T19:36:42.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com'
2023-06-14T19:36:43.225 INFO:teuthology.misc:Compressing logs...
2023-06-14T19:36:43.260 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-06-14T19:36:43.260 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-06-14T19:36:43.261 INFO:teuthology.orchestra.run.smithi151.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	/home/ubuntu/cephtest/archive/syslog/misc.log:	 77.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-06-14T19:36:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-06-14T19:36:43.294 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-06-14T19:36:43.295 INFO:teuthology.orchestra.run.smithi191.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	 77.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-06-14T19:36:43.644 INFO:teuthology.orchestra.run.smithi191.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-06-14T19:36:43.645 INFO:teuthology.orchestra.run.smithi191.stderr: 92.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-06-14T19:36:45.553 INFO:teuthology.orchestra.run.smithi151.stderr: 94.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-06-14T19:36:45.963 INFO:teuthology.misc:Compressing logs...
2023-06-14T19:36:46.012 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2023-06-14T19:36:46.013 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/cephadm.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-volume.log
2023-06-14T19:36:46.014 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.2.log
2023-06-14T19:36:46.018 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-volume.log:	/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.2.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi151.swhzyr.log
2023-06-14T19:36:46.029 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2023-06-14T19:36:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-volume.log
2023-06-14T19:36:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/cephadm.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi191.qwgfty.log
2023-06-14T19:36:46.030 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.3.log
2023-06-14T19:36:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-volume.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mgr.smithi191.ytejxd.log
2023-06-14T19:36:46.031 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi191.qwgfty.log:	/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.3.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mon.smithi191.log
2023-06-14T19:36:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mgr.smithi191.ytejxd.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi191.rpndkh.log
2023-06-14T19:36:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mon.smithi191.log:	/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi191.rpndkh.log:	 89.6% -- replaced with /var/log/ceph/cephadm.log.gz
2023-06-14T19:36:46.037 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.4.log
2023-06-14T19:36:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.4.log:	 91.4% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-volume.log.gz
2023-06-14T19:36:46.041 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.5.log
2023-06-14T19:36:46.048 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi151.swhzyr.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.0.log
2023-06-14T19:36:46.056 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.0.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mgr.smithi151.swzbyv.log
2023-06-14T19:36:46.078 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mgr.smithi151.swzbyv.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mon.smithi151.log
2023-06-14T19:36:46.078 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mon.smithi151.log:	gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.1.log
2023-06-14T19:36:46.132 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi151.rlxtnk.log
2023-06-14T19:36:46.178 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.5.log:	 94.9% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi191.rpndkh.log.gz
2023-06-14T19:36:46.248 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.1.log:	/var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi151.rlxtnk.log:	 94.9% -- replaced with /var/log/ceph/cephadm.log.gz
2023-06-14T19:36:46.275 INFO:teuthology.orchestra.run.smithi151.stderr: 94.9% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi151.swhzyr.log.gz
2023-06-14T19:36:47.753 INFO:teuthology.orchestra.run.smithi151.stderr: 92.2% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-volume.log.gz
2023-06-14T19:36:54.613 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mgr.smithi151.swzbyv.log: file size changed while zipping
2023-06-14T19:36:54.613 INFO:teuthology.orchestra.run.smithi151.stderr: 89.4% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mgr.smithi151.swzbyv.log.gz
2023-06-14T19:36:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi151.rlxtnk.log: file size changed while zipping
2023-06-14T19:36:57.616 INFO:teuthology.orchestra.run.smithi151.stderr: 95.0% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi151.rlxtnk.log.gz
2023-06-14T19:37:00.661 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2023-06-14T19:37:00.661 INFO:teuthology.orchestra.run.smithi191.stderr: 93.7% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.5.log.gz
2023-06-14T19:37:01.245 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mgr.smithi191.ytejxd.log: file size changed while zipping
2023-06-14T19:37:01.246 INFO:teuthology.orchestra.run.smithi191.stderr: 86.6% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mgr.smithi191.ytejxd.log.gz
2023-06-14T19:37:04.065 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2023-06-14T19:37:04.066 INFO:teuthology.orchestra.run.smithi191.stderr: 93.6% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.4.log.gz
2023-06-14T19:37:05.661 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2023-06-14T19:37:05.662 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.2.log.gz
2023-06-14T19:37:06.319 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2023-06-14T19:37:06.320 INFO:teuthology.orchestra.run.smithi151.stderr: 93.6% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.0.log.gz
2023-06-14T19:37:07.920 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mon.smithi191.log: file size changed while zipping
2023-06-14T19:37:07.920 INFO:teuthology.orchestra.run.smithi191.stderr: 92.3% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mon.smithi191.log.gz
2023-06-14T19:37:08.596 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2023-06-14T19:37:08.596 INFO:teuthology.orchestra.run.smithi151.stderr: 93.6% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.1.log.gz
2023-06-14T19:37:11.661 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2023-06-14T19:37:11.662 INFO:teuthology.orchestra.run.smithi191.stderr: 93.8% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-osd.3.log.gz
2023-06-14T19:37:19.678 INFO:teuthology.orchestra.run.smithi151.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mon.smithi151.log: file size changed while zipping
2023-06-14T19:37:19.678 INFO:teuthology.orchestra.run.smithi151.stderr: 90.7% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mon.smithi151.log.gz
2023-06-14T19:38:08.398 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi191.qwgfty.log: file size changed while zipping
2023-06-14T19:38:08.398 INFO:teuthology.orchestra.run.smithi191.stderr: 93.0% -- replaced with /var/log/ceph/ff6dccbc-0a87-11ee-9b2b-001a4aab830c/ceph-mds.cephfs.smithi191.qwgfty.log.gz
2023-06-14T19:38:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:tar: ./ff6dccbc-0a87-11ee-9b2b-001a4aab830c: file changed as we read it
2023-06-14T19:38:39.082 INFO:teuthology.kill:No teuthology processes running
2023-06-14T19:38:39.921 INFO:teuthology.kill:Nuking machines: ['smithi151', 'smithi191']
2023-06-14T19:38:40.555 INFO:teuthology.kill:2023-06-14 19:38:40,555.555 INFO:teuthology.nuke:targets:
2023-06-14T19:38:40.556 ERROR:teuthology.dispatcher.supervisor:Failed to kill job and unlock machines
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/dispatcher/supervisor.py", line 320, in run_with_watchdog
    kill.kill_job(
  File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/kill.py", line 101, in kill_job
    nuke_targets(targets, owner)
  File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/kill.py", line 276, in nuke_targets
    sys.stdout.flush()
OSError: [Errno 5] Input/output error
2023-06-14T19:40:40.805 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-06-14T19:40:40.915 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi151 but it was locked by another job: /home/teuthworker/archive/adking-2023-06-14_18:35:42-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi/7304510
2023-06-14T19:40:40.915 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi191 but it was locked by another job: /home/teuthworker/archive/adking-2023-06-14_18:35:42-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi/7304510