2022-03-13T07:41:23.429 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-13T07:41:23.568 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi064.front.sepia.ceph.com' reimaging 2022-03-13T07:41:23.570 INFO:teuthology.lock.ops:Updating [ubuntu@smithi064.front.sepia.ceph.com]: reset os type and version on server 2022-03-13T07:41:23.570 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2022-03-13T07:41:23.590 INFO:teuthology.lock.ops:Node 'ubuntu@smithi064.front.sepia.ceph.com' reimaging is complete 2022-03-13T07:41:23.590 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging 2022-03-13T07:41:23.590 INFO:teuthology.lock.ops:Updating [ubuntu@smithi106.front.sepia.ceph.com]: reset os type and version on server 2022-03-13T07:41:23.591 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2022-03-13T07:41:23.610 INFO:teuthology.lock.ops:Node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging is complete 2022-03-13T07:41:23.803 INFO:teuthology.provision.fog.smithi064:Scheduling deploy of rhel 8.4 2022-03-13T07:41:23.829 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of rhel 8.4 2022-03-13T07:41:24.151 INFO:teuthology.orchestra.console:Power off smithi064 2022-03-13T07:41:24.202 INFO:teuthology.orchestra.console:Power off smithi106 2022-03-13T07:41:32.537 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2022-03-13T07:41:32.638 INFO:teuthology.orchestra.console:Power on smithi064 2022-03-13T07:41:36.680 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2022-03-13T07:41:36.781 INFO:teuthology.orchestra.console:Power on smithi106 2022-03-13T07:41:41.038 INFO:teuthology.orchestra.console:Power on for smithi106 completed 2022-03-13T07:41:41.110 INFO:teuthology.orchestra.console:Power on for smithi064 completed 2022-03-13T07:41:41.139 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish 2022-03-13T07:41:41.211 INFO:teuthology.provision.fog.smithi064:Waiting for deploy to finish 2022-03-13T07:46:13.699 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-13T07:46:13.920 INFO:teuthology.provision.fog.smithi064:Node is ready 2022-03-13T07:46:13.937 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064.front.sepia.ceph.com 2022-03-13T07:46:13.994 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64 smithi064.front.sepia.ceph.com smithi064 2022-03-13T07:46:14.320 INFO:teuthology.provision.fog.smithi064:Deploy complete! 2022-03-13T07:46:28.568 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-13T07:46:28.807 INFO:teuthology.provision.fog.smithi106:Node is ready 2022-03-13T07:46:28.824 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com 2022-03-13T07:46:28.880 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106 2022-03-13T07:46:29.210 INFO:teuthology.provision.fog.smithi106:Deploy complete! 2022-03-13T07:46:29.364 INFO:teuthology.lock.ops:Checking smithi064.front.sepia.ceph.com 2022-03-13T07:46:29.365 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-13T07:46:29.385 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com 2022-03-13T07:46:29.386 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-13T07:46:29.404 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: set os type and version on server 2022-03-13T07:46:29.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-03-13T07:46:29.643 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2022-03-13T07:46:29.698 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Red Hat Enterprise Linux" 2022-03-13T07:46:29.699 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8.4 (Ootpa)" 2022-03-13T07:46:29.699 INFO:teuthology.orchestra.run.smithi064.stdout:ID="rhel" 2022-03-13T07:46:29.699 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="fedora" 2022-03-13T07:46:29.700 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8.4" 2022-03-13T07:46:29.700 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2022-03-13T07:46:29.700 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-03-13T07:46:29.700 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2022-03-13T07:46:29.701 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-03-13T07:46:29.701 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.redhat.com/" 2022-03-13T07:46:29.701 INFO:teuthology.orchestra.run.smithi064.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-03-13T07:46:29.701 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-13T07:46:29.701 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-03-13T07:46:29.702 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-13T07:46:29.702 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-03-13T07:46:29.702 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-03-13T07:46:29.702 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-03-13T07:46:29.704 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2022-03-13T07:46:29.722 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server 2022-03-13T07:46:29.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2022-03-13T07:46:29.949 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2022-03-13T07:46:30.004 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Red Hat Enterprise Linux" 2022-03-13T07:46:30.004 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8.4 (Ootpa)" 2022-03-13T07:46:30.005 INFO:teuthology.orchestra.run.smithi106.stdout:ID="rhel" 2022-03-13T07:46:30.005 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="fedora" 2022-03-13T07:46:30.005 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8.4" 2022-03-13T07:46:30.005 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8" 2022-03-13T07:46:30.006 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-03-13T07:46:30.006 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2022-03-13T07:46:30.006 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-03-13T07:46:30.006 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.redhat.com/" 2022-03-13T07:46:30.007 INFO:teuthology.orchestra.run.smithi106.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-03-13T07:46:30.007 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-13T07:46:30.007 INFO:teuthology.orchestra.run.smithi106.stdout: 2022-03-13T07:46:30.007 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-13T07:46:30.007 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-03-13T07:46:30.008 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-03-13T07:46:30.008 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-03-13T07:46:30.009 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2022-03-13T07:46:30.106 INFO:teuthology.dispatcher.supervisor:Running job 6734376 2022-03-13T07:46:30.107 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-03-13_03:31:03-rados-pacific-distro-default-smithi/6734376 --name teuthology-2022-03-13_03:31:03-rados-pacific-distro-default-smithi --description rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} -- /home/teuthworker/archive/teuthology-2022-03-13_03:31:03-rados-pacific-distro-default-smithi/6734376/orig.config.yaml 2022-03-13T07:46:30.110 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-03-13_03:31:03-rados-pacific-distro-default-smithi/6734376 2022-03-13T07:46:30.111 INFO:teuthology.dispatcher.supervisor:Job PID: 14760 2022-03-13T07:46:30.111 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-13T14:16:59.083 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-13T14:16:59.249 INFO:teuthology.kill:Killing Pids: {14760} 2022-03-13T14:17:01.083 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2022-03-13T14:17:01.084 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2022-03-13T14:17:01.085 INFO:teuthology.misc:Compressing logs... 2022-03-13T14:17:01.085 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-03-13T14:17:01.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2022-03-13T14:17:01.569 INFO:teuthology.misc:Compressing logs... 2022-03-13T14:17:01.805 INFO:teuthology.misc:Compressing logs... 2022-03-13T14:17:12.600 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-03-13T14:17:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-03-13T14:17:15.485 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-03-13T14:17:17.747 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-03-13T14:17:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-03-13T14:17:28.676 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-03-13T14:17:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-03-13T14:17:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-03-13T14:17:46.043 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-03-13T14:17:51.082 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-03-13T14:17:54.845 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/b07fc7c8-a2a4-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-03-13T14:18:15.593 INFO:teuthology.kill:No teuthology processes running 2022-03-13T14:18:16.611 INFO:teuthology.kill:Nuking machines: ['smithi064', 'smithi106'] 2022-03-13T14:18:17.516 INFO:teuthology.kill:2022-03-13 14:18:17,516.516 INFO:teuthology.nuke:targets: 2022-03-13T14:18:17.517 INFO:teuthology.kill: smithi064.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDreDPeGS6DmtHlP1PkLXBdTWyY4b24ZYlLRsdmq7Lqh8Z+s+Y6PMZ00IKTVvw6V36yB+DzzppCJY+yA/ePEd2W0fXqjkHJ7QXpZxZBc0qloPQMEMttlkp6FXBV3j2z55y04HNPICF7PzWlXTKD7mCegz86eUX+LAItzH5gAfrW8JFeA5nIyWFiJU6fOjGc6sA9BnlMLCzL+UQpes+Is2niaa+Gp+0AcgfQGdDMsyhDouugr7n9MTjlOygnx2BDfUS/wRaVhwmxSshrHPzYnCvIlzdbBt2SymIXAgy6HxcMJaENi9ZSxPWoyChXCEXT/waGze8l5qRy1q0+rnsAqYgmYU/x4zy64qe3mukMKnByT3N5WmLYT3HQDRHlgP1xxpXcsVI6O9W4biHtDnQIza9EMRdkUsRIbLQHrHFBXpZBJmzowf9OSIs84EqrumGGJIySutps3jzN6fPWLY8+ZLSPMdw8rocI61P5e1KzwcCMd/xVBOJzmWidAqZGA3LDPz0= 2022-03-13T14:18:17.517 INFO:teuthology.kill: smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC46fBhmeoPUsizc+KoJexBH0xI4P4QbF/x9M4rJeruuCjJn4xKyj6nopkB2cQZ8SN1JZueihvDEgAbf9QzW1HOde4VNmk8yH0ZMAlPVVm+d8z0ZLW6jP5Qrb4XlweF1ORyN/Gf/goRDeTa8anWrxBX75P0IlS9/6y47G99Lxgfejm39D+PF49kg7Ui68y0cTUaR5U4gHIMpwiNz3k9ofIbshYcrzyeckjWSsi5/++CViVOvEq+nxkFS/ZPnQkt9rqEDsw2fJqD2+VSN+Vknple5nBL3hhOnkYAR9ZOzdgMGwyPayxkyjKGIHlDVEtvdojGdSuHmMACDVXWsT9hxHLEXw01cWVMUvfogrOGn3ZXKX6HWeyjLhh+VgpTOGV/qsiakIsnJ3ojAVJGTJWE8ZaDl6EfSwIyN0Og+dtpR6KNmpgVW1FE4nP3i1jdHitGEToM8LlPGyxLhar+LyTLHkx/MYGmQAOK2v7Wr+wHwX6QcKaI6N8YArqYg1JoUXk3F/c= 2022-03-13T14:18:17.543 INFO:teuthology.kill:2022-03-13 14:18:17,543.543 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-13T14:18:17.546 INFO:teuthology.kill:2022-03-13 14:18:17,546.546 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-13T14:18:17.624 INFO:teuthology.kill:2022-03-13 14:18:17,624.624 INFO:teuthology.orchestra.console:Power off smithi106 2022-03-13T14:18:17.635 INFO:teuthology.kill:2022-03-13 14:18:17,635.635 INFO:teuthology.orchestra.console:Power off smithi064 2022-03-13T14:18:25.995 INFO:teuthology.kill:2022-03-13 14:18:25,994.994 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2022-03-13T14:18:26.001 INFO:teuthology.kill:2022-03-13 14:18:26,001.001 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2022-03-13T14:18:26.167 INFO:teuthology.kill:2022-03-13 14:18:26,166.166 INFO:teuthology.lock.ops:unlocked: smithi064.front.sepia.ceph.com 2022-03-13T14:18:26.171 INFO:teuthology.kill:2022-03-13 14:18:26,171.171 INFO:teuthology.lock.ops:unlocked: smithi106.front.sepia.ceph.com 2022-03-13T14:20:26.428 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15