2023-05-02T14:28:45.731 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-05-02T14:28:45.877 INFO:teuthology.lock.ops:Start node 'smithi064.front.sepia.ceph.com' reimaging 2023-05-02T14:28:45.879 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: reset os type and version on server 2023-05-02T14:28:45.879 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-05-02T14:28:45.906 INFO:teuthology.lock.ops:Node 'smithi064.front.sepia.ceph.com' reimaging is complete 2023-05-02T14:28:45.907 INFO:teuthology.lock.ops:Start node 'smithi081.front.sepia.ceph.com' reimaging 2023-05-02T14:28:45.907 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: reset os type and version on server 2023-05-02T14:28:45.907 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2023-05-02T14:28:45.931 INFO:teuthology.lock.ops:Node 'smithi081.front.sepia.ceph.com' reimaging is complete 2023-05-02T14:28:46.068 INFO:teuthology.provision.fog.smithi064:Scheduling deploy of centos 8.stream 2023-05-02T14:28:46.078 INFO:teuthology.provision.fog.smithi081:Scheduling deploy of centos 8.stream 2023-05-02T14:28:46.316 INFO:teuthology.orchestra.console:Power off smithi064 2023-05-02T14:28:46.333 INFO:teuthology.orchestra.console:Power off smithi081 2023-05-02T14:28:54.699 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2023-05-02T14:28:54.705 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2023-05-02T14:28:54.801 INFO:teuthology.orchestra.console:Power on smithi064 2023-05-02T14:28:54.810 INFO:teuthology.orchestra.console:Power on smithi081 2023-05-02T14:29:03.180 INFO:teuthology.orchestra.console:Power on for smithi064 completed 2023-05-02T14:29:03.189 INFO:teuthology.orchestra.console:Power on for smithi081 completed 2023-05-02T14:29:03.282 INFO:teuthology.provision.fog.smithi064:Waiting for deploy to finish 2023-05-02T14:29:03.290 INFO:teuthology.provision.fog.smithi081:Waiting for deploy to finish 2023-05-02T14:31:52.293 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-05-02T14:31:52.294 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-05-02T14:32:01.349 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-05-02T14:32:01.350 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-05-02T14:32:10.441 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-05-02T14:32:10.442 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-05-02T14:32:19.525 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-05-02T14:32:19.526 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-05-02T14:32:28.581 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-05-02T14:32:28.582 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-05-02T14:32:37.641 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-05-02T14:32:37.642 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-05-02T14:32:46.729 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-05-02T14:33:02.213 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-05-02T14:33:11.269 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-05-02T14:33:17.622 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-05-02T14:33:24.488 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-05-02T14:33:42.499 INFO:teuthology.provision.fog.smithi081:Node is ready 2023-05-02T14:33:42.520 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081.front.sepia.ceph.com 2023-05-02T14:33:42.577 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81 smithi081.front.sepia.ceph.com smithi081 2023-05-02T14:33:42.930 INFO:teuthology.provision.fog.smithi081:Deploy complete! 2023-05-02T14:33:44.672 INFO:teuthology.provision.fog.smithi064:Node is ready 2023-05-02T14:33:44.695 INFO:teuthology.orchestra.run.smithi064.stdout:smithi001 2023-05-02T14:33:44.755 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.1 smithi001.front.sepia.ceph.com smithi001 2023-05-02T14:33:44.756 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.1 smithi001 smithi001 2023-05-02T14:33:45.154 INFO:teuthology.provision.fog.smithi064:Deploy complete! 2023-05-02T14:33:45.347 INFO:teuthology.lock.ops:Checking smithi064.front.sepia.ceph.com 2023-05-02T14:33:45.348 INFO:teuthology.lock.ops:New key found. Updating... 2023-05-02T14:33:45.368 INFO:teuthology.lock.ops:Checking smithi081.front.sepia.ceph.com 2023-05-02T14:33:45.369 INFO:teuthology.lock.ops:New key found. Updating... 2023-05-02T14:33:45.394 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: set os type and version on server 2023-05-02T14:33:45.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-05-02T14:33:45.588 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-05-02T14:33:45.646 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2023-05-02T14:33:45.647 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8" 2023-05-02T14:33:45.647 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2023-05-02T14:33:45.647 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2023-05-02T14:33:45.647 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8" 2023-05-02T14:33:45.647 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2023-05-02T14:33:45.647 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 8" 2023-05-02T14:33:45.647 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2023-05-02T14:33:45.647 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-05-02T14:33:45.648 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2023-05-02T14:33:45.648 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-05-02T14:33:45.648 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-05-02T14:33:45.648 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-05-02T14:33:45.649 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-05-02T14:33:45.678 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: set os type and version on server 2023-05-02T14:33:45.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-05-02T14:33:45.875 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2023-05-02T14:33:45.934 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2023-05-02T14:33:45.935 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="8" 2023-05-02T14:33:45.935 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2023-05-02T14:33:45.935 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2023-05-02T14:33:45.935 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="8" 2023-05-02T14:33:45.935 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el8" 2023-05-02T14:33:45.935 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 8" 2023-05-02T14:33:45.935 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2023-05-02T14:33:45.936 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-05-02T14:33:45.936 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2023-05-02T14:33:45.936 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-05-02T14:33:45.936 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-05-02T14:33:45.936 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-05-02T14:33:45.937 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2023-05-02T14:33:46.053 INFO:teuthology.dispatcher.supervisor:Running job 7253632 2023-05-02T14:33:46.053 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2023-04-26_00:39:50-rgw-wip-cbodley2-testing-distro-default-smithi/7253632 --name cbodley-2023-04-26_00:39:50-rgw-wip-cbodley2-testing-distro-default-smithi --description rgw/upgrade/{1-install/quincy/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides} -- /home/teuthworker/archive/cbodley-2023-04-26_00:39:50-rgw-wip-cbodley2-testing-distro-default-smithi/7253632/orig.config.yaml 2023-05-02T14:33:46.057 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2023-04-26_00:39:50-rgw-wip-cbodley2-testing-distro-default-smithi/7253632 2023-05-02T14:33:46.058 INFO:teuthology.dispatcher.supervisor:Job PID: 12692 2023-05-02T14:33:46.058 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-05-03T02:34:43.145 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-05-03T02:34:43.290 INFO:teuthology.kill:Killing Pids: {12692} 2023-05-03T02:34:44.124 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-05-03T02:34:44.125 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-05-03T02:34:44.126 INFO:teuthology.misc:Compressing logs... 2023-05-03T02:34:44.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-05-03T02:34:44.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-05-03T02:34:44.621 INFO:teuthology.misc:Compressing logs... 2023-05-03T02:34:44.650 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-03T02:34:44.651 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-03T02:34:44.651 INFO:teuthology.orchestra.run.smithi064.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 81.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-05-03T02:34:44.660 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-05-03T02:34:44.688 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-03T02:34:44.689 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-03T02:34:44.689 INFO:teuthology.orchestra.run.smithi081.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 81.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-05-03T02:34:44.697 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-05-03T02:34:44.767 INFO:teuthology.misc:Compressing logs... 2023-05-03T02:34:44.798 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113592.log 2023-05-03T02:34:44.799 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114151.log 2023-05-03T02:34:44.799 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113592.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-05-03T02:34:44.801 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113592.log.gz 2023-05-03T02:34:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114151.log.gz 2023-05-03T02:34:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-05-03T02:34:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114263.log 2023-05-03T02:34:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114160.log 2023-05-03T02:34:44.801 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107750.log 2023-05-03T02:34:44.802 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114263.log.gz 2023-05-03T02:34:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114813.log 2023-05-03T02:34:44.802 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114160.log.gz 2023-05-03T02:34:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107750.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2023-05-03T02:34:44.802 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107750.log.gz 2023-05-03T02:34:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114114.log 2023-05-03T02:34:44.802 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114813.log.gz 2023-05-03T02:34:44.802 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114474.log 2023-05-03T02:34:44.805 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113858.log 2023-05-03T02:34:44.806 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114114.log.gz 2023-05-03T02:34:44.806 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114474.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-05-03T02:34:44.806 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114474.log.gz 2023-05-03T02:34:44.806 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2023-05-03T02:34:44.806 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113543.log 2023-05-03T02:34:44.806 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113858.log.gz 2023-05-03T02:34:44.806 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113486.log 2023-05-03T02:34:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113543.log.gz 2023-05-03T02:34:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-05-03T02:34:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113865.log 2023-05-03T02:34:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113486.log.gz 2023-05-03T02:34:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114764.log 2023-05-03T02:34:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113865.log.gz 2023-05-03T02:34:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.101022.log 2023-05-03T02:34:44.807 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114764.log.gz 2023-05-03T02:34:44.808 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113848.log 2023-05-03T02:34:44.808 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.tmp-client.admin.101022.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.101022.log.gz 2023-05-03T02:34:44.808 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113848.log.gz 2023-05-03T02:34:44.808 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113862.log 2023-05-03T02:34:44.808 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113863.log 2023-05-03T02:34:44.808 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113862.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-05-03T02:34:44.808 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113862.log.gz 2023-05-03T02:34:44.809 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113863.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-05-03T02:34:44.809 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113863.log.gz 2023-05-03T02:34:44.810 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113866.log 2023-05-03T02:34:44.810 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.113866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113866.log.gz 2023-05-03T02:34:44.810 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-05-03T02:34:44.810 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.3.log: 94.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-05-03T02:34:44.811 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114572.log 2023-05-03T02:34:44.812 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114572.log.gz 2023-05-03T02:34:44.812 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113641.log 2023-05-03T02:34:44.813 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113641.log.gz 2023-05-03T02:34:44.813 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114717.log 2023-05-03T02:34:44.814 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114717.log.gz 2023-05-03T02:34:44.814 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114668.log 2023-05-03T02:34:44.814 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114668.log.gz 2023-05-03T02:34:44.815 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114268.log 2023-05-03T02:34:44.816 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114268.log.gz 2023-05-03T02:34:44.816 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-05-03T02:34:44.822 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114264.log 2023-05-03T02:34:44.823 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114264.log.gz 2023-05-03T02:34:44.823 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-05-03T02:34:44.834 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107341.log 2023-05-03T02:34:44.835 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107612.log 2023-05-03T02:34:44.837 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113255.log 2023-05-03T02:34:44.837 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107193.log 2023-05-03T02:34:44.837 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.1.log 2023-05-03T02:34:44.838 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107444.log 2023-05-03T02:34:44.838 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107341.log.gz 2023-05-03T02:34:44.838 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113255.log.gz 2023-05-03T02:34:44.838 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107193.log.gz 2023-05-03T02:34:44.838 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/rgw.ceph.client.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-05-03T02:34:44.838 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113396.log 2023-05-03T02:34:44.838 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107444.log.gz 2023-05-03T02:34:44.838 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-05-03T02:34:44.839 INFO:teuthology.orchestra.run.smithi081.stderr: 94.3% -- replaced with /var/log/ceph/rgw.ceph.client.1.log.gz 2023-05-03T02:34:44.839 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113349.log 2023-05-03T02:34:44.839 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113396.log.gz 2023-05-03T02:34:44.839 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107388.log 2023-05-03T02:34:44.839 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113349.log.gz 2023-05-03T02:34:44.839 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-05-03T02:34:44.839 INFO:teuthology.orchestra.run.smithi081.stderr:gzip /var/log/ceph/ceph-client.admin.107388.log: -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-05-03T02:34:44.840 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107388.log.gz 2023-05-03T02:34:44.840 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-05-03T02:34:44.840 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113302.log 2023-05-03T02:34:44.844 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.113302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113302.log.gz 2023-05-03T02:34:44.844 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-05-03T02:34:44.844 INFO:teuthology.orchestra.run.smithi081.stderr: 90.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-05-03T02:34:44.844 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-05-03T02:34:44.847 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107612.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107612.log.gz 2023-05-03T02:34:44.847 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-05-03T02:34:44.848 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107243.log 2023-05-03T02:34:44.849 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107243.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107243.log.gz 2023-05-03T02:34:44.849 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107495.log 2023-05-03T02:34:44.850 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107495.log.gz 2023-05-03T02:34:44.850 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107145.log 2023-05-03T02:34:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107145.log.gz 2023-05-03T02:34:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107293.log 2023-05-03T02:34:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107293.log.gz 2023-05-03T02:34:44.858 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114621.log 2023-05-03T02:34:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114621.log.gz 2023-05-03T02:34:44.880 INFO:teuthology.orchestra.run.smithi081.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-05-03T02:34:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107252.log 2023-05-03T02:34:44.885 INFO:teuthology.orchestra.run.smithi064.stderr: 95.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-05-03T02:34:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.107252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107252.log.gz 2023-05-03T02:34:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114259.log 2023-05-03T02:34:44.888 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114259.log.gz 2023-05-03T02:34:44.888 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113368.log 2023-05-03T02:34:44.890 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113368.log.gz 2023-05-03T02:34:44.890 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113870.log 2023-05-03T02:34:44.892 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113870.log.gz 2023-05-03T02:34:44.892 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113872.log 2023-05-03T02:34:44.893 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113872.log.gz 2023-05-03T02:34:44.893 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114159.log 2023-05-03T02:34:44.894 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114159.log.gz 2023-05-03T02:34:44.894 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113272.log 2023-05-03T02:34:44.894 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113272.log.gz 2023-05-03T02:34:44.895 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114523.log 2023-05-03T02:34:44.895 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.114523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114523.log.gz 2023-05-03T02:34:44.895 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113437.log 2023-05-03T02:34:44.896 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.113437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113437.log.gz 2023-05-03T02:34:45.258 INFO:teuthology.orchestra.run.smithi064.stderr: 95.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-05-03T02:34:50.432 INFO:teuthology.orchestra.run.smithi081.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-05-03T02:34:53.067 INFO:teuthology.orchestra.run.smithi081.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-05-03T02:34:55.103 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-05-03T02:35:02.230 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-05-03T02:35:05.774 INFO:teuthology.orchestra.run.smithi081.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-05-03T02:35:05.910 INFO:teuthology.orchestra.run.smithi081.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-05-03T02:35:06.556 INFO:teuthology.orchestra.run.smithi064.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-05-03T02:35:06.571 INFO:teuthology.orchestra.run.smithi081.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-05-03T02:35:09.807 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-05-03T02:35:11.442 INFO:teuthology.orchestra.run.smithi081.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-05-03T02:35:12.094 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-05-03T02:35:13.002 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-05-03T02:35:37.357 INFO:teuthology.kill:No teuthology processes running 2023-05-03T02:35:38.123 INFO:teuthology.kill:Nuking machines: ['smithi064', 'smithi081'] 2023-05-03T02:35:38.714 INFO:teuthology.kill:2023-05-03 02:35:38,714.714 INFO:teuthology.nuke:targets: 2023-05-03T02:35:38.715 INFO:teuthology.kill: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCLadOQ82ebRFvM32XoB/QflrN5onHQ4GynhvVecPxbbwhm56ZH+ZcXnDd12vgHc0PfkHs+q3nh+7w2wdSYWdpI= 2023-05-03T02:35:38.715 INFO:teuthology.kill: smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKpub/IWTzHAD7lOtx9nRnRvqN8Zp7x7NKRKgAswCHhfNUVpHAJAtmWHEvPi+yMkhKUOwzX3aq1DTYHpjU1oR18= 2023-05-03T02:35:38.715 INFO:teuthology.kill:2023-05-03 02:35:38,714.714 INFO:teuthology.nuke:Checking targets against current locks 2023-05-03T02:35:38.768 INFO:teuthology.kill:2023-05-03 02:35:38,768.768 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-03T02:35:38.799 INFO:teuthology.kill:2023-05-03 02:35:38,799.799 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-03T02:35:38.853 INFO:teuthology.kill:2023-05-03 02:35:38,852.852 INFO:teuthology.orchestra.console:Power off smithi064 2023-05-03T02:35:38.897 INFO:teuthology.kill:2023-05-03 02:35:38,897.897 INFO:teuthology.orchestra.console:Power off smithi081 2023-05-03T02:35:47.234 INFO:teuthology.kill:2023-05-03 02:35:47,234.234 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2023-05-03T02:35:47.261 INFO:teuthology.kill:2023-05-03 02:35:47,261.261 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2023-05-03T02:35:47.392 INFO:teuthology.kill:2023-05-03 02:35:47,391.391 INFO:teuthology.lock.ops:unlocked: smithi064.front.sepia.ceph.com 2023-05-03T02:35:47.417 INFO:teuthology.kill:2023-05-03 02:35:47,416.416 INFO:teuthology.lock.ops:unlocked: smithi081.front.sepia.ceph.com 2023-05-03T02:37:47.747 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-05-03T02:37:47.846 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi064 but it was locked by another job: /home/teuthworker/archive/teuthology-2023-04-29_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7257989 2023-05-03T02:37:47.847 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi081 but it was locked by another job: /home/teuthworker/archive/teuthology-2023-04-29_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7257989