2022-04-23T20:30:08.397 INFO:root:teuthology version: 1.1.1.dev313+g1b11219b
2022-04-23T20:30:08.521 INFO:teuthology.lock.ops:Start node 'smithi005.front.sepia.ceph.com' reimaging
2022-04-23T20:30:08.522 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: reset os type and version on server
2022-04-23T20:30:08.522 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2022-04-23T20:30:08.541 INFO:teuthology.lock.ops:Node 'smithi005.front.sepia.ceph.com' reimaging is complete
2022-04-23T20:30:08.541 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging
2022-04-23T20:30:08.542 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server
2022-04-23T20:30:08.542 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2022-04-23T20:30:08.561 INFO:teuthology.lock.ops:Node 'smithi125.front.sepia.ceph.com' reimaging is complete
2022-04-23T20:30:08.752 INFO:teuthology.provision.fog.smithi005:Scheduling deploy of centos 8.stream
2022-04-23T20:30:08.772 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of centos 8.stream
2022-04-23T20:30:09.093 INFO:teuthology.orchestra.console:Power off smithi125
2022-04-23T20:30:09.105 INFO:teuthology.orchestra.console:Power off smithi005
2022-04-23T20:30:17.466 INFO:teuthology.orchestra.console:Power off for smithi005 completed
2022-04-23T20:30:17.567 INFO:teuthology.orchestra.console:Power on smithi005
2022-04-23T20:30:21.573 INFO:teuthology.orchestra.console:Power off for smithi125 completed
2022-04-23T20:30:21.675 INFO:teuthology.orchestra.console:Power on smithi125
2022-04-23T20:30:21.813 INFO:teuthology.orchestra.console:Power on for smithi005 completed
2022-04-23T20:30:21.914 INFO:teuthology.provision.fog.smithi005:Waiting for deploy to finish
2022-04-23T20:30:30.037 INFO:teuthology.orchestra.console:Power on for smithi125 completed
2022-04-23T20:30:30.138 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish
2022-04-23T20:34:03.611 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-23T20:34:23.405 INFO:teuthology.provision.fog.smithi005:Node is ready
2022-04-23T20:34:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005.front.sepia.ceph.com
2022-04-23T20:34:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5 smithi005.front.sepia.ceph.com smithi005
2022-04-23T20:34:23.835 INFO:teuthology.provision.fog.smithi005:Deploy complete!
2022-04-23T20:34:33.525 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-23T20:34:59.056 INFO:teuthology.provision.fog.smithi125:Node is ready
2022-04-23T20:34:59.074 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com
2022-04-23T20:34:59.130 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125
2022-04-23T20:34:59.560 INFO:teuthology.provision.fog.smithi125:Deploy complete!
2022-04-23T20:34:59.767 INFO:teuthology.lock.ops:Checking smithi005.front.sepia.ceph.com
2022-04-23T20:34:59.767 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-23T20:34:59.791 INFO:teuthology.lock.ops:Checking smithi125.front.sepia.ceph.com
2022-04-23T20:34:59.791 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-23T20:34:59.810 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: set os type and version on server
2022-04-23T20:34:59.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com'
2022-04-23T20:35:00.117 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64
2022-04-23T20:35:00.177 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream"
2022-04-23T20:35:00.177 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/"
2022-04-23T20:35:00.178 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-23T20:35:00.179 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-23T20:35:00.179 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-23T20:35:00.180 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2022-04-23T20:35:00.198 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: set os type and version on server
2022-04-23T20:35:00.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com'
2022-04-23T20:35:00.388 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64
2022-04-23T20:35:00.444 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream"
2022-04-23T20:35:00.444 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="8"
2022-04-23T20:35:00.444 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos"
2022-04-23T20:35:00.444 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="8"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el8"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-23T20:35:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-23T20:35:00.446 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2022-04-23T20:35:00.547 INFO:teuthology.dispatcher.supervisor:Running job 6802531
2022-04-23T20:35:00.547 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_1b11219b9af33884aae5eb66700541df4b9620ed/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-04-23_14:46:12-upgrade-master-distro-default-smithi/6802531 --name yuriw-2022-04-23_14:46:12-upgrade-master-distro-default-smithi --description upgrade/pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} -- /home/teuthworker/archive/yuriw-2022-04-23_14:46:12-upgrade-master-distro-default-smithi/6802531/orig.config.yaml
2022-04-23T20:35:00.551 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-04-23_14:46:12-upgrade-master-distro-default-smithi/6802531
2022-04-23T20:35:00.551 INFO:teuthology.dispatcher.supervisor:Job PID: 12649
2022-04-23T20:35:00.552 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-24T03:05:29.363 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-04-24T03:05:29.559 INFO:teuthology.kill:Killing Pids: {12649}
2022-04-24T03:05:31.047 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2022-04-24T03:05:31.063 INFO:teuthology.task.internal:roles: ubuntu@smithi125.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2022-04-24T03:05:31.063 INFO:teuthology.misc:Compressing logs...
2022-04-24T03:05:31.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com'
2022-04-24T03:05:31.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com'
2022-04-24T03:05:31.739 INFO:teuthology.misc:Compressing logs...
2022-04-24T03:05:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-04-24T03:05:32.296 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-04-24T03:05:32.532 INFO:teuthology.misc:Compressing logs...
2022-04-24T03:07:58.456 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-04-24T03:08:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2022-04-24T03:09:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2022-04-24T03:10:03.917 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-04-24T03:10:09.174 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2022-04-24T03:11:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-04-24T03:13:04.039 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-04-24T03:15:46.571 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-04-24T03:15:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-04-24T03:18:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-04-24T03:18:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/3691b1a2-c346-11ec-8c39-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2022-04-24T03:23:31.209 INFO:teuthology.kill:No teuthology processes running
2022-04-24T03:23:32.526 INFO:teuthology.kill:Nuking machines: ['smithi005', 'smithi125']
2022-04-24T03:23:33.749 INFO:teuthology.kill:2022-04-24 03:23:33,749.749 INFO:teuthology.nuke:targets:
2022-04-24T03:23:33.749 INFO:teuthology.kill:  smithi005.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFJZhm0WJhrld+0uJ3WcbCy2C0RKaOK+wYrfqOIWS2dRnUXOBZ6fpAui2FX/whD/fnpIll26O0X8HU7KIgBs9+7DtOfuCTdIB1BJWH6jKbmfpsEJ7iBoTTz+aAhHaqd3xzMNUIHSJIJuP4NCn+RS8Ik99X1CnR/7Rv2liDlNkGNi9jqp/KfCX9MJc3ESilq7EG3Doohi60ZlLl1Ea7NaDBgrL/v9+gZFeJUuZQW17IfV3FpJsBWzzVD/9FSWftM0XGxQPVIZYVkXoZYI8xeKIPRsRlQOPKWVHWt85KtFkTNWKf+jv2u3xcHFXV2I1cwOTsGJjI3XmmzoQwm1AGGNwk86aXRll/BtMvxuE1dn3ctl0wlafxcZywYiSVIDheNBIImK6YhtIYjzxNNTHSZbjl85kYyt45WeDDl/RU/8u4AZnlIOSklpoz7X5VUTNl7tFgTP2Q7MUk7UpcD8lfeAah6FvNocGrJHctqRekq1dnuRVApZhZ7Z/2B085kVb3O1c=
2022-04-24T03:23:33.750 INFO:teuthology.kill:  smithi125.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDg9VP4j4hay7zB25fgeiyljTNmxQEanOSoMpkHxCeNuGFcbJ782CpJBPj/OrZECilkDvEWPX+ZI6B5L5T4wHPIKnPDH47zLLAZKLBtiTg6TaA22/8xJdbSdKpFsxobPp53CfKYTHhEBW5s7eHNskf+9NQXm5iWVVFLj5lSGzYEDHZDRKiH3l1TS+dZYmuqauRc1iYe828Ayn0lINaFuw706NBR23FpuiuJsfXKWoMsd3betNGsoWzATURV86kxPCJRkEMtJt/H1YOD3GcQ2JJrry8ly8CD7RBp0YD3etf5gAXQ5rIYX6E72Do7vsBPfPJWjRdFvCVBIb4Nrlsi3rHalGKS1+QNMZJVod0DosqE9mU+xK5F9ku65vm/+jGBSy2l48WiBZ/Bt3YDrK8i4XXugo2hP/VCzx9ebSInq+taZJIRnnu3RXwTtnfYiZ2zJjXMxy6phGc2IjRl7f+dpYdkCHhEQ2PQ9GEug38dYYNeLvr3FVPf69r1ysBscowBAvs=
2022-04-24T03:23:33.778 INFO:teuthology.kill:2022-04-24 03:23:33,777.777 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-24T03:23:33.782 INFO:teuthology.kill:2022-04-24 03:23:33,781.781 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-24T03:23:33.866 INFO:teuthology.kill:2022-04-24 03:23:33,865.865 INFO:teuthology.orchestra.console:Power off smithi005
2022-04-24T03:23:33.878 INFO:teuthology.kill:2022-04-24 03:23:33,877.877 INFO:teuthology.orchestra.console:Power off smithi125
2022-04-24T03:23:42.240 INFO:teuthology.kill:2022-04-24 03:23:42,239.239 INFO:teuthology.orchestra.console:Power off for smithi125 completed
2022-04-24T03:23:42.384 INFO:teuthology.kill:2022-04-24 03:23:42,383.383 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com
2022-04-24T03:23:46.347 INFO:teuthology.kill:2022-04-24 03:23:46,347.347 INFO:teuthology.orchestra.console:Power off for smithi005 completed
2022-04-24T03:23:46.493 INFO:teuthology.kill:2022-04-24 03:23:46,493.493 INFO:teuthology.lock.ops:unlocked: smithi005.front.sepia.ceph.com
2022-04-24T03:25:46.772 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-24T03:25:46.918 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi005 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-23_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6802418
2022-04-24T03:25:46.918 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi125 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-23_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6802418