2022-04-01T23:46:42.761 INFO:root:teuthology version: 1.1.0-d877b023 2022-04-01T23:46:42.880 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi099.front.sepia.ceph.com' reimaging 2022-04-01T23:46:42.881 INFO:teuthology.lock.ops:Updating [ubuntu@smithi099.front.sepia.ceph.com]: reset os type and version on server 2022-04-01T23:46:42.881 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2022-04-01T23:46:42.898 INFO:teuthology.lock.ops:Node 'ubuntu@smithi099.front.sepia.ceph.com' reimaging is complete 2022-04-01T23:46:42.899 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi201.front.sepia.ceph.com' reimaging 2022-04-01T23:46:42.899 INFO:teuthology.lock.ops:Updating [ubuntu@smithi201.front.sepia.ceph.com]: reset os type and version on server 2022-04-01T23:46:42.899 INFO:teuthology.lock.ops:Updating smithi201.front.sepia.ceph.com on lock server 2022-04-01T23:46:42.919 INFO:teuthology.lock.ops:Node 'ubuntu@smithi201.front.sepia.ceph.com' reimaging is complete 2022-04-01T23:46:43.106 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of centos 8.stream 2022-04-01T23:46:43.128 INFO:teuthology.provision.fog.smithi201:Scheduling deploy of centos 8.stream 2022-04-01T23:46:43.464 INFO:teuthology.orchestra.console:Power off smithi099 2022-04-01T23:46:43.522 INFO:teuthology.orchestra.console:Power off smithi201 2022-04-01T23:46:51.883 INFO:teuthology.orchestra.console:Power off for smithi201 completed 2022-04-01T23:46:51.985 INFO:teuthology.orchestra.console:Power on smithi201 2022-04-01T23:46:55.937 INFO:teuthology.orchestra.console:Power off for smithi099 completed 2022-04-01T23:46:56.039 INFO:teuthology.orchestra.console:Power on smithi099 2022-04-01T23:47:00.344 INFO:teuthology.orchestra.console:Power on for smithi201 completed 2022-04-01T23:47:00.445 INFO:teuthology.provision.fog.smithi201:Waiting for deploy to finish 2022-04-01T23:47:04.403 INFO:teuthology.orchestra.console:Power on for smithi099 completed 2022-04-01T23:47:04.504 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish 2022-04-01T23:51:06.908 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-01T23:51:18.767 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-01T23:51:32.214 INFO:teuthology.provision.fog.smithi201:Node is ready 2022-04-01T23:51:32.236 INFO:teuthology.orchestra.run.smithi201.stdout:smithi201.front.sepia.ceph.com 2022-04-01T23:51:32.295 INFO:teuthology.orchestra.run.smithi201.stdout:172.21.15.201 smithi201.front.sepia.ceph.com smithi201 2022-04-01T23:51:32.669 INFO:teuthology.provision.fog.smithi201:Deploy complete! 2022-04-01T23:51:43.704 INFO:teuthology.provision.fog.smithi099:Node is ready 2022-04-01T23:51:43.726 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com 2022-04-01T23:51:43.785 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099 2022-04-01T23:51:44.159 INFO:teuthology.provision.fog.smithi099:Deploy complete! 2022-04-01T23:51:44.347 INFO:teuthology.lock.ops:Checking smithi099.front.sepia.ceph.com 2022-04-01T23:51:44.348 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-01T23:51:44.369 INFO:teuthology.lock.ops:Checking smithi201.front.sepia.ceph.com 2022-04-01T23:51:44.369 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-01T23:51:44.388 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: set os type and version on server 2022-04-01T23:51:44.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-04-01T23:51:44.574 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2022-04-01T23:51:44.631 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2022-04-01T23:51:44.632 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8" 2022-04-01T23:51:44.632 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2022-04-01T23:51:44.632 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2022-04-01T23:51:44.632 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8" 2022-04-01T23:51:44.632 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2022-04-01T23:51:44.633 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 8" 2022-04-01T23:51:44.633 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2022-04-01T23:51:44.633 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-04-01T23:51:44.633 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2022-04-01T23:51:44.634 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-04-01T23:51:44.634 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-04-01T23:51:44.634 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-04-01T23:51:44.635 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2022-04-01T23:51:44.661 INFO:teuthology.lock.ops:Updating [smithi201.front.sepia.ceph.com]: set os type and version on server 2022-04-01T23:51:44.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi201.front.sepia.ceph.com' 2022-04-01T23:51:44.910 INFO:teuthology.orchestra.run.smithi201.stdout:x86_64 2022-04-01T23:51:44.969 INFO:teuthology.orchestra.run.smithi201.stdout:NAME="CentOS Stream" 2022-04-01T23:51:44.970 INFO:teuthology.orchestra.run.smithi201.stdout:VERSION="8" 2022-04-01T23:51:44.970 INFO:teuthology.orchestra.run.smithi201.stdout:ID="centos" 2022-04-01T23:51:44.970 INFO:teuthology.orchestra.run.smithi201.stdout:ID_LIKE="rhel fedora" 2022-04-01T23:51:44.970 INFO:teuthology.orchestra.run.smithi201.stdout:VERSION_ID="8" 2022-04-01T23:51:44.971 INFO:teuthology.orchestra.run.smithi201.stdout:PLATFORM_ID="platform:el8" 2022-04-01T23:51:44.971 INFO:teuthology.orchestra.run.smithi201.stdout:PRETTY_NAME="CentOS Stream 8" 2022-04-01T23:51:44.971 INFO:teuthology.orchestra.run.smithi201.stdout:ANSI_COLOR="0;31" 2022-04-01T23:51:44.971 INFO:teuthology.orchestra.run.smithi201.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-04-01T23:51:44.972 INFO:teuthology.orchestra.run.smithi201.stdout:HOME_URL="https://centos.org/" 2022-04-01T23:51:44.972 INFO:teuthology.orchestra.run.smithi201.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-04-01T23:51:44.972 INFO:teuthology.orchestra.run.smithi201.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-04-01T23:51:44.972 INFO:teuthology.orchestra.run.smithi201.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-04-01T23:51:44.973 INFO:teuthology.lock.ops:Updating smithi201.front.sepia.ceph.com on lock server 2022-04-01T23:51:45.073 INFO:teuthology.dispatcher.supervisor:Running job 6773213 2022-04-01T23:51:45.073 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-04-01_18:40:00-orch-wip-yuri11-testing-2022-04-01-0842-distro-default-smithi/6773213 --name yuriw-2022-04-01_18:40:00-orch-wip-yuri11-testing-2022-04-01-0842-distro-default-smithi --description orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} -- /home/teuthworker/archive/yuriw-2022-04-01_18:40:00-orch-wip-yuri11-testing-2022-04-01-0842-distro-default-smithi/6773213/orig.config.yaml 2022-04-01T23:51:45.077 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-04-01_18:40:00-orch-wip-yuri11-testing-2022-04-01-0842-distro-default-smithi/6773213 2022-04-01T23:51:45.078 INFO:teuthology.dispatcher.supervisor:Job PID: 2689 2022-04-01T23:51:45.078 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-04-02T06:22:14.439 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-04-02T06:22:14.598 INFO:teuthology.kill:Killing Pids: {2689} 2022-04-02T06:22:15.781 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2022-04-02T06:22:15.782 INFO:teuthology.task.internal:roles: ubuntu@smithi201.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2022-04-02T06:22:15.782 INFO:teuthology.misc:Compressing logs... 2022-04-02T06:22:15.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-04-02T06:22:15.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi201.front.sepia.ceph.com' 2022-04-02T06:22:16.264 INFO:teuthology.misc:Compressing logs... 2022-04-02T06:22:16.530 INFO:teuthology.misc:Compressing logs... 2022-04-02T06:22:18.638 INFO:teuthology.orchestra.run.smithi201.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-mgr.x.log: file size changed while zipping 2022-04-02T06:22:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-04-02T06:22:29.182 INFO:teuthology.orchestra.run.smithi201.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-04-02T06:22:36.934 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-04-02T06:22:40.392 INFO:teuthology.orchestra.run.smithi201.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-04-02T06:22:49.851 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-04-02T06:22:50.064 INFO:teuthology.orchestra.run.smithi201.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-04-02T06:22:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-04-02T06:22:55.624 INFO:teuthology.orchestra.run.smithi201.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-04-02T06:23:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-04-02T06:23:07.581 INFO:teuthology.orchestra.run.smithi201.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-04-02T06:23:21.437 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/60f667ce-b218-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-04-02T06:23:46.151 INFO:teuthology.kill:No teuthology processes running 2022-04-02T06:23:47.289 INFO:teuthology.kill:Nuking machines: ['smithi099', 'smithi201'] 2022-04-02T06:23:48.254 INFO:teuthology.kill:2022-04-02 06:23:48,253.253 INFO:teuthology.nuke:targets: 2022-04-02T06:23:48.254 INFO:teuthology.kill: smithi099.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoXyNNu56GBhvKMtqeGl6D0X/a86EoujP3wewWFSWDN7JMKxlJVBv/CVrCJC5t0Xu58e4VLvSe+4PZ5LuqFuRiA+CPNEhRkKXd6NBHz/TJMJftd1H8BdT0Rq9iGvoWredeRga/Ngtq8ktX62fqWGu5vjbPez17rXYtb3P6gxqlk/edeffSI/8iOPU4YGeP1JaK9wfUOHVR5/WmoRpYI2Xv2mDiZnJ+mqKeQwn0I47Cd2co8Eq5EBjym+aFgxaFoMVk+dZHVRBGou4sS4BcOZ1Ob6lKiHlkRDGeswu/lpjft/ewoG1McWJi2sxJgkQLJjJlWby3eGGQX3MKJSHos52ccBzoGj/r//tGFjsjJIXXNfLm6M7lGam3cgj3JnTWqVpI1BExIWX1ECMB7BejEZJ6Jdf4IFsLUhIQa8SOWscF8MB0ZUqrRC2flHOfsM5mGvU4d+BjMnlMr0Pf9rZXqZ04UB33R1g0Bvwxcof+Z5ZthiOfm5rGHW/5kcFygy/SPQ8= 2022-04-02T06:23:48.255 INFO:teuthology.kill: smithi201.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC69ZlTD1nhdBHgKdhyh26hcgsryw7HcLqYh3XloPkKDS1D9A+aJUqGuVBDDH9sX93pOttC6vUTt8OFzG6zA6c3kca2KWXeT7TvPto3AoXLpInEUVnocTLKpdIchYgKG8AmSTXCicAjPkX3inm1x2Iu/bdo1vmIvdU/wGJR58FylQQHZzNOvpa9nm+dbQTHzU2CZ7PFyn2fR5XoQR49sD9DgpOlQ36iz28WkzcrzK7qpD1tQ+Tqd4EKbUckaBDWyKli4mJjrdKFeJ4JQLnAJOHeeSmm556tQ4PaVNdqzq+8v6AIDnLsRf+9uCUXWsN1EtLNaNk421tHdAKF5dwg+NILi9glbs+NVObZGHY8i1QPLs0Eield9i9afMw2MaUbXVxk5oZ2shUO7ykzV6ysjyyMGVHIKn+4ZtnhCUPnc037JH1m7K0ihlT63dVRkhl6jYenhm6mQAnhN7S7zRJxpv7nLj1PNisFuoV8lhGJQhF5r6k9L36T2A68St62w2iXoAs= 2022-04-02T06:23:48.280 INFO:teuthology.kill:2022-04-02 06:23:48,280.280 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-02T06:23:48.283 INFO:teuthology.kill:2022-04-02 06:23:48,283.283 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-02T06:23:48.356 INFO:teuthology.kill:2022-04-02 06:23:48,356.356 INFO:teuthology.orchestra.console:Power off smithi099 2022-04-02T06:23:48.369 INFO:teuthology.kill:2022-04-02 06:23:48,369.369 INFO:teuthology.orchestra.console:Power off smithi201 2022-04-02T06:23:56.736 INFO:teuthology.kill:2022-04-02 06:23:56,735.735 INFO:teuthology.orchestra.console:Power off for smithi201 completed 2022-04-02T06:23:56.882 INFO:teuthology.kill:2022-04-02 06:23:56,881.881 INFO:teuthology.lock.ops:unlocked: smithi201.front.sepia.ceph.com 2022-04-02T06:24:00.840 INFO:teuthology.kill:2022-04-02 06:24:00,840.840 INFO:teuthology.orchestra.console:Power off for smithi099 completed 2022-04-02T06:24:00.986 INFO:teuthology.kill:2022-04-02 06:24:00,986.986 INFO:teuthology.lock.ops:unlocked: smithi099.front.sepia.ceph.com 2022-04-02T06:26:01.324 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-04-02T06:26:01.442 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi099 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-01_05:17:03-krbd-pacific-testing-default-smithi/6771859 2022-04-02T06:26:01.442 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi201 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-01_05:17:03-krbd-pacific-testing-default-smithi/6771859