2023-04-04T17:43:31.963 INFO:root:teuthology version: 0.0.1.dev93+gd7c4062 2023-04-04T17:43:32.107 INFO:teuthology.lock.ops:Start node 'smithi045.front.sepia.ceph.com' reimaging 2023-04-04T17:43:32.108 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: reset os type and version on server 2023-04-04T17:43:32.109 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-04-04T17:43:32.130 INFO:teuthology.lock.ops:Node 'smithi045.front.sepia.ceph.com' reimaging is complete 2023-04-04T17:43:32.130 INFO:teuthology.lock.ops:Start node 'smithi077.front.sepia.ceph.com' reimaging 2023-04-04T17:43:32.130 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: reset os type and version on server 2023-04-04T17:43:32.131 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-04-04T17:43:32.154 INFO:teuthology.lock.ops:Node 'smithi077.front.sepia.ceph.com' reimaging is complete 2023-04-04T17:43:32.278 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of centos 8.stream 2023-04-04T17:43:32.310 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of centos 8.stream 2023-04-04T17:43:32.474 INFO:teuthology.orchestra.console:Power off smithi045 2023-04-04T17:43:32.531 INFO:teuthology.orchestra.console:Power off smithi077 2023-04-04T17:43:32.621 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2023-04-04T17:43:32.670 INFO:teuthology.orchestra.console:Power off for smithi077 completed 2023-04-04T17:43:32.722 INFO:teuthology.orchestra.console:Power on smithi045 2023-04-04T17:43:32.772 INFO:teuthology.orchestra.console:Power on smithi077 2023-04-04T17:43:41.088 INFO:teuthology.orchestra.console:Power on for smithi045 completed 2023-04-04T17:43:41.135 INFO:teuthology.orchestra.console:Power on for smithi077 completed 2023-04-04T17:43:41.189 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish 2023-04-04T17:43:41.236 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish 2023-04-04T17:46:30.149 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-04-04T17:46:30.150 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-04-04T17:46:39.205 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-04-04T17:46:39.206 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-04-04T17:46:48.261 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-04-04T17:46:48.293 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-04-04T17:46:57.317 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-04-04T17:46:57.349 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-04-04T17:47:06.373 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-04-04T17:47:06.409 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-04-04T17:47:15.429 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-04-04T17:47:24.489 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-04-04T17:48:02.261 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-04-04T17:48:12.411 WARNING:teuthology.provision.fog:timed out 2023-04-04T17:48:18.751 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-04-04T17:48:19.013 INFO:teuthology.provision.fog.smithi045:Node is ready 2023-04-04T17:48:19.072 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com 2023-04-04T17:48:19.131 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045 2023-04-04T17:48:19.497 INFO:teuthology.provision.fog.smithi045:Deploy complete! 2023-04-04T17:48:22.772 INFO:teuthology.provision.fog.smithi077:Node is ready 2023-04-04T17:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com 2023-04-04T17:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077 2023-04-04T17:48:23.237 INFO:teuthology.provision.fog.smithi077:Deploy complete! 2023-04-04T17:48:23.412 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com 2023-04-04T17:48:23.413 INFO:teuthology.lock.ops:New key found. Updating... 2023-04-04T17:48:23.439 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com 2023-04-04T17:48:23.439 INFO:teuthology.lock.ops:New key found. Updating... 2023-04-04T17:48:23.457 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server 2023-04-04T17:48:23.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-04-04T17:48:23.656 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2023-04-04T17:48:23.718 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2023-04-04T17:48:23.718 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8" 2023-04-04T17:48:23.718 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2023-04-04T17:48:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2023-04-04T17:48:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8" 2023-04-04T17:48:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2023-04-04T17:48:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 8" 2023-04-04T17:48:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-04-04T17:48:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-04-04T17:48:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2023-04-04T17:48:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-04-04T17:48:23.720 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-04-04T17:48:23.720 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-04-04T17:48:23.721 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-04-04T17:48:23.747 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server 2023-04-04T17:48:23.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-04-04T17:48:23.937 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2023-04-04T17:48:23.995 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2023-04-04T17:48:23.995 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8" 2023-04-04T17:48:23.995 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2023-04-04T17:48:23.996 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2023-04-04T17:48:23.996 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8" 2023-04-04T17:48:23.996 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8" 2023-04-04T17:48:23.996 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 8" 2023-04-04T17:48:23.996 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2023-04-04T17:48:23.996 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-04-04T17:48:23.997 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2023-04-04T17:48:23.997 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-04-04T17:48:23.997 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-04-04T17:48:23.997 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-04-04T17:48:23.998 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-04-04T17:48:24.113 INFO:teuthology.dispatcher.supervisor:Running job 7231026 2023-04-04T17:48:24.113 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_d7c4062c8e48ef4dca373feb7285bdfb014e59b7/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-04-04_15:06:57-fs-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7231026 --name yuriw-2023-04-04_15:06:57-fs-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2023-04-04_15:06:57-fs-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7231026/orig.config.yaml 2023-04-04T17:48:24.116 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-04-04_15:06:57-fs-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7231026 2023-04-04T17:48:24.117 INFO:teuthology.dispatcher.supervisor:Job PID: 17135 2023-04-04T17:48:24.117 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-04-05T05:49:21.993 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-04-05T05:49:22.145 INFO:teuthology.kill:Killing Pids: {17135} 2023-04-05T05:49:23.159 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2023-04-05T05:49:23.160 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2023-04-05T05:49:23.160 INFO:teuthology.misc:Compressing logs... 2023-04-05T05:49:23.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-04-05T05:49:23.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-04-05T05:49:23.641 INFO:teuthology.misc:Compressing logs... 2023-04-05T05:49:23.680 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2023-04-05T05:49:23.681 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2023-04-05T05:49:23.681 INFO:teuthology.orchestra.run.smithi045.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 86.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-04-05T05:49:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2023-04-05T05:49:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2023-04-05T05:49:23.712 INFO:teuthology.orchestra.run.smithi077.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 78.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-04-05T05:49:23.797 INFO:teuthology.orchestra.run.smithi077.stderr: 91.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-04-05T05:49:23.812 INFO:teuthology.orchestra.run.smithi045.stderr: 93.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-04-05T05:49:23.941 INFO:teuthology.misc:Compressing logs... 2023-04-05T05:49:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-04-05T05:49:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-volume.log 2023-04-05T05:49:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.2.log 2023-04-05T05:49:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: 0.0% -- replaced with /var/log/ceph/cephadm.log.gz 2023-04-05T05:49:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.1.log 2023-04-05T05:49:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi045.iyhvvd.log 2023-04-05T05:49:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.2.log: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi045.trjldr.log 2023-04-05T05:49:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi045.iyhvvd.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.0.log 2023-04-05T05:49:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi045.trjldr.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mgr.smithi045.zrfeij.log 2023-04-05T05:49:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.0.log: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mgr.smithi045.zrfeij.log: 90.3% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-volume.log.gz 2023-04-05T05:49:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mon.smithi045.log 2023-04-05T05:49:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-04-05T05:49:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-volume.log 2023-04-05T05:49:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.5.log 2023-04-05T05:49:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mon.smithi077.log 2023-04-05T05:49:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.3.log 2023-04-05T05:49:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.5.log: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mon.smithi077.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi077.vugknz.log 2023-04-05T05:49:24.011 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi077.ttvnvh.log 2023-04-05T05:49:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi077.vugknz.log: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi077.ttvnvh.log: 89.0% -- replaced with /var/log/ceph/cephadm.log.gz 2023-04-05T05:49:24.012 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.4.log 2023-04-05T05:49:24.012 INFO:teuthology.orchestra.run.smithi077.stderr: 89.0% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-volume.log.gz 2023-04-05T05:49:24.014 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mgr.smithi077.brsvdt.log 2023-04-05T05:49:24.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mon.smithi045.log: 95.1% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi045.trjldr.log.gz 2023-04-05T05:49:24.093 INFO:teuthology.orchestra.run.smithi045.stderr: 95.1% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi045.iyhvvd.log.gz 2023-04-05T05:49:24.165 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mgr.smithi077.brsvdt.log: 93.8% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mgr.smithi077.brsvdt.log.gz 2023-04-05T05:49:27.490 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mgr.smithi045.zrfeij.log: file size changed while zipping 2023-04-05T05:49:27.490 INFO:teuthology.orchestra.run.smithi045.stderr: 92.4% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mgr.smithi045.zrfeij.log.gz 2023-04-05T05:49:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mon.smithi077.log: file size changed while zipping 2023-04-05T05:49:30.750 INFO:teuthology.orchestra.run.smithi077.stderr: 93.5% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mon.smithi077.log.gz 2023-04-05T05:49:34.556 INFO:teuthology.orchestra.run.smithi077.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi077.vugknz.log: file size changed while zipping 2023-04-05T05:49:34.557 INFO:teuthology.orchestra.run.smithi077.stderr: 95.0% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi077.vugknz.log.gz 2023-04-05T05:49:35.891 INFO:teuthology.orchestra.run.smithi077.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2023-04-05T05:49:35.891 INFO:teuthology.orchestra.run.smithi077.stderr: 93.4% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.4.log.gz 2023-04-05T05:49:37.628 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2023-04-05T05:49:37.628 INFO:teuthology.orchestra.run.smithi045.stderr: 93.6% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.0.log.gz 2023-04-05T05:49:38.218 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2023-04-05T05:49:38.219 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.2.log.gz 2023-04-05T05:49:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mon.smithi045.log: file size changed while zipping 2023-04-05T05:49:38.337 INFO:teuthology.orchestra.run.smithi045.stderr: 89.8% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mon.smithi045.log.gz 2023-04-05T05:49:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2023-04-05T05:49:39.576 INFO:teuthology.orchestra.run.smithi045.stderr: 93.8% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.1.log.gz 2023-04-05T05:49:41.283 INFO:teuthology.orchestra.run.smithi077.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2023-04-05T05:49:41.283 INFO:teuthology.orchestra.run.smithi077.stderr: 93.6% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.5.log.gz 2023-04-05T05:49:44.849 INFO:teuthology.orchestra.run.smithi077.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2023-04-05T05:49:44.849 INFO:teuthology.orchestra.run.smithi077.stderr: 93.3% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-osd.3.log.gz 2023-04-05T05:50:45.913 INFO:teuthology.orchestra.run.smithi077.stderr:gzip: /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi077.ttvnvh.log: file size changed while zipping 2023-04-05T05:50:45.913 INFO:teuthology.orchestra.run.smithi077.stderr: 93.1% -- replaced with /var/log/ceph/d19c9354-d31e-11ed-9aff-001a4aab830c/ceph-mds.cephfs.smithi077.ttvnvh.log.gz 2023-04-05T05:51:09.561 INFO:teuthology.kill:No teuthology processes running 2023-04-05T05:51:10.376 INFO:teuthology.kill:Nuking machines: ['smithi045', 'smithi077'] 2023-04-05T05:51:11.145 INFO:teuthology.kill:2023-04-05 05:51:11,144.144 INFO:teuthology.nuke:targets: 2023-04-05T05:51:11.145 INFO:teuthology.kill: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRI7IDD7W6wBZTcmXO8TfUmNz3adCNp20AI4xperCwrwIJZrScLHURQQb1kFRKdhJfSRTNQcBuF2pbRiOq3O0s= 2023-04-05T05:51:11.146 INFO:teuthology.kill: smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqN0dOLsZL+f0sFYccsm/GW8lySEoamGAUtr+fclKnqI43MP6rLl3Kzl6E47OP6GvE/F+gE96HWwp1zzqYsFRgOVdLLrgAye1eRDjQATsCE9oOp8ErTy8yW5am9R44E1oGHWcW1MH1mAc6glZja4hvGAZWpHzzLYNrGuc+Sbbfq0UP6kswXM/eY5HXuGNM8ZX/N5LlpuDoq8MxpdcH28Q73q3k+M5a619po91azLGu2AMWhL+8J8Gz6OdGRTCrimP/XLdrpvsbuwfvKL0UP1d4ZS02W2L1hFffBwKkqjNEe4SD6DU85cV4Hz3bmvUBCJQsWF94k3mxzrg2kJMRoQBPIIkD/058hThwQRYW6EOLx0jyHYf6bl/AQMPZ6rqdZ1G6agwI3Izz/LCzsLgxp+v7zzFKFPDBeCNOqpq3TXf9hXLxs0Mop+ezzQpPEn+7sQkifR1qq2fFw/ASgKz7OZagKR0hbHe2wLjF3wE3M012blXO10soB22sVKjo5loHrEU= 2023-04-05T05:51:11.146 INFO:teuthology.kill:2023-04-05 05:51:11,144.144 INFO:teuthology.nuke:Checking targets against current locks 2023-04-05T05:51:11.206 INFO:teuthology.kill:2023-04-05 05:51:11,206.206 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-04-05T05:51:11.232 INFO:teuthology.kill:2023-04-05 05:51:11,232.232 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-04-05T05:51:11.296 INFO:teuthology.kill:2023-04-05 05:51:11,296.296 INFO:teuthology.orchestra.console:Power off smithi045 2023-04-05T05:51:11.335 INFO:teuthology.kill:2023-04-05 05:51:11,335.335 INFO:teuthology.orchestra.console:Power off smithi077 2023-04-05T05:51:23.821 INFO:teuthology.kill:2023-04-05 05:51:23,821.821 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2023-04-05T05:51:23.831 INFO:teuthology.kill:2023-04-05 05:51:23,831.831 INFO:teuthology.orchestra.console:Power off for smithi077 completed 2023-04-05T05:51:23.977 INFO:teuthology.kill:2023-04-05 05:51:23,977.977 INFO:teuthology.lock.ops:unlocked: smithi045.front.sepia.ceph.com 2023-04-05T05:51:23.993 INFO:teuthology.kill:2023-04-05 05:51:23,993.993 INFO:teuthology.lock.ops:unlocked: smithi077.front.sepia.ceph.com 2023-04-05T05:53:24.335 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-04-05T05:53:24.474 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi045 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-04-04_15:24:40-rados-wip-yuri4-testing-2023-03-31-1237-distro-default-smithi/7231428 2023-04-05T05:53:24.474 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi077 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-04-04_15:24:40-rados-wip-yuri4-testing-2023-03-31-1237-distro-default-smithi/7231428