2022-09-29T05:09:27.019 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9
2022-09-29T05:09:27.548 INFO:teuthology.lock.ops:Start node 'smithi006.front.sepia.ceph.com' reimaging
2022-09-29T05:09:27.557 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: reset os type and version on server
2022-09-29T05:09:27.558 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server
2022-09-29T05:09:27.638 INFO:teuthology.lock.ops:Node 'smithi006.front.sepia.ceph.com' reimaging is complete
2022-09-29T05:09:27.639 INFO:teuthology.lock.ops:Start node 'smithi120.front.sepia.ceph.com' reimaging
2022-09-29T05:09:27.639 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: reset os type and version on server
2022-09-29T05:09:27.639 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2022-09-29T05:09:27.757 INFO:teuthology.lock.ops:Node 'smithi120.front.sepia.ceph.com' reimaging is complete
2022-09-29T05:09:28.228 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of rhel 8.6
2022-09-29T05:09:28.373 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.6
2022-09-29T05:09:28.941 INFO:teuthology.orchestra.console:Power off smithi006
2022-09-29T05:09:29.302 INFO:teuthology.orchestra.console:Power off smithi120
2022-09-29T05:09:37.496 INFO:teuthology.orchestra.console:Power off for smithi006 completed
2022-09-29T05:09:37.603 INFO:teuthology.orchestra.console:Power on smithi006
2022-09-29T05:09:38.056 INFO:teuthology.orchestra.console:Power off for smithi120 completed
2022-09-29T05:09:38.177 INFO:teuthology.orchestra.console:Power on smithi120
2022-09-29T05:09:42.106 INFO:teuthology.orchestra.console:Power on for smithi006 completed
2022-09-29T05:09:42.212 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish
2022-09-29T05:09:46.752 INFO:teuthology.orchestra.console:Power on for smithi120 completed
2022-09-29T05:09:46.862 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish
2022-09-29T05:14:02.249 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-29T05:14:02.527 INFO:teuthology.provision.fog.smithi006:Node is ready
2022-09-29T05:14:02.552 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com
2022-09-29T05:14:02.612 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006
2022-09-29T05:14:02.969 INFO:teuthology.provision.fog.smithi006:Deploy complete!
2022-09-29T05:14:24.940 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-29T05:14:25.211 INFO:teuthology.provision.fog.smithi120:Node is ready
2022-09-29T05:14:25.270 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com
2022-09-29T05:14:25.329 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120
2022-09-29T05:14:25.668 INFO:teuthology.provision.fog.smithi120:Deploy complete!
2022-09-29T05:14:25.887 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com
2022-09-29T05:14:25.888 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-29T05:14:25.915 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com
2022-09-29T05:14:25.916 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-29T05:14:25.938 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server
2022-09-29T05:14:25.939 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com'
2022-09-29T05:14:26.215 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64
2022-09-29T05:14:26.274 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Red Hat Enterprise Linux"
2022-09-29T05:14:26.275 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8.6 (Ootpa)"
2022-09-29T05:14:26.275 INFO:teuthology.orchestra.run.smithi006.stdout:ID="rhel"
2022-09-29T05:14:26.275 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="fedora"
2022-09-29T05:14:26.276 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8.6"
2022-09-29T05:14:26.277 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8"
2022-09-29T05:14:26.277 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-09-29T05:14:26.277 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31"
2022-09-29T05:14:26.278 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-09-29T05:14:26.278 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.redhat.com/"
2022-09-29T05:14:26.278 INFO:teuthology.orchestra.run.smithi006.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-09-29T05:14:26.278 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-09-29T05:14:26.279 INFO:teuthology.orchestra.run.smithi006.stdout:
2022-09-29T05:14:26.279 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-09-29T05:14:26.279 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-09-29T05:14:26.279 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-09-29T05:14:26.279 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-09-29T05:14:26.281 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server
2022-09-29T05:14:26.301 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server
2022-09-29T05:14:26.301 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2022-09-29T05:14:26.505 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64
2022-09-29T05:14:26.562 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux"
2022-09-29T05:14:26.563 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.6 (Ootpa)"
2022-09-29T05:14:26.563 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel"
2022-09-29T05:14:26.563 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora"
2022-09-29T05:14:26.564 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.6"
2022-09-29T05:14:26.564 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8"
2022-09-29T05:14:26.564 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-09-29T05:14:26.564 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31"
2022-09-29T05:14:26.564 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-09-29T05:14:26.565 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/"
2022-09-29T05:14:26.565 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-09-29T05:14:26.565 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-09-29T05:14:26.565 INFO:teuthology.orchestra.run.smithi120.stdout:
2022-09-29T05:14:26.566 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-09-29T05:14:26.566 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-09-29T05:14:26.566 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-09-29T05:14:26.566 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-09-29T05:14:26.567 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2022-09-29T05:14:26.685 INFO:teuthology.dispatcher.supervisor:Running job 7047305
2022-09-29T05:14:26.686 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-09-29_05:03:04-upgrade:telemetry-upgrade-main-distro-default-smithi/7047305 --name lflores-2022-09-29_05:03:04-upgrade:telemetry-upgrade-main-distro-default-smithi --description upgrade:telemetry-upgrade/quincy-x/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks} -- /home/teuthworker/archive/lflores-2022-09-29_05:03:04-upgrade:telemetry-upgrade-main-distro-default-smithi/7047305/orig.config.yaml
2022-09-29T05:14:26.693 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-09-29_05:03:04-upgrade:telemetry-upgrade-main-distro-default-smithi/7047305
2022-09-29T05:14:26.695 INFO:teuthology.dispatcher.supervisor:Job PID: 24991
2022-09-29T05:14:26.696 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-09-29T17:15:20.291 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-09-29T17:15:20.921 INFO:teuthology.kill:Killing Pids: {24991}
2022-09-29T17:15:22.515 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2022-09-29T17:15:22.518 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mds.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2022-09-29T17:15:22.525 INFO:teuthology.misc:Compressing logs...
2022-09-29T17:15:22.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com'
2022-09-29T17:15:23.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2022-09-29T17:15:24.003 INFO:teuthology.misc:Compressing logs...
2022-09-29T17:15:24.457 INFO:teuthology.misc:Compressing logs...
2022-09-29T17:15:38.609 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-09-29T17:15:40.306 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-09-29T17:15:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-09-29T17:15:56.927 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-09-29T17:16:08.259 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-09-29T17:16:13.832 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-09-29T17:16:25.687 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-09-29T17:16:31.245 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-09-29T17:16:32.925 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2022-09-29T17:16:43.421 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2022-09-29T17:17:07.898 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2022-09-29T17:17:13.899 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/53a3ed9e-3fb7-11ed-8431-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2022-09-29T17:17:50.389 INFO:teuthology.kill:No teuthology processes running
2022-09-29T17:17:51.854 INFO:teuthology.kill:Nuking machines: ['smithi006', 'smithi120']
2022-09-29T17:17:53.041 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-09-29T17:17:53.042 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-09-29T17:17:53.042 INFO:teuthology.kill:2022-09-29 17:17:53,041.041 INFO:teuthology.nuke:targets:
2022-09-29T17:17:53.043 INFO:teuthology.kill:  smithi006.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCl/b8KOAzhl4WiQBb+SCmS/kHkxfMY9gWeGltzBC5/aTKA/ib/BC5W1yoh/uaBJGPUc2qhV3ja5fbtxRWRIeVNe8dbcL50riyfxowx33fL0o71HrQdTFtZNKepy2o0ya4GWF7LLdD1IqFWSZAExUY9Ll4pQmjWdDPJaK41gnP2V8P7CkmtiQJnoG0AYP9+al7tMt+g5FDVyU8XiwFv1+lV57PPc+tnLGiiY8MgOnkgUCWojGa/9xgZJtXS1+FL9r91Bm8kWqjjUO7BSHWrBabdT5IsWKZHDps0uf6tMYlCcue7CDgB6cyWHeyUs8e8UIrWVaz+GffppY6cd80L1epbTmNTR0Rh4BUemckT9H0uhp0Ma+S3vuL6UornrIokTl93Cfd+ILKvR8YbM3a/7lXcRANMokbxlEoucxUJnGfAfg04ydef2qnQvvBrDal3PjqqSvd7Kt0/KtV5u53R2xa64Nw6eePweImhhJ9CPXLWeQQnnLEPlmq7soT7YEJstm0=
2022-09-29T17:17:53.043 INFO:teuthology.kill:  smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqJlTgo1ScD0NgMQms/zQb3VAMJiIoIPX79+qDFAakeRm/j7bubyePJO3z36iQgb2VMgg07DGGIFsNXjC9w/xfBz+jaB+AQscTLofpgbWO1Hl5RaQ+LKJxe+vV3ASWvoBxjMQh4+cQv4ahi+8HdMnpbVMWwwaI8ftzBBlBi3Qha0GvLXixwrRsHNGWktdY0pAPoM0cwo3uBJkBVOpGKwgSv+ohtymZFY4V4PY7WEN69WAeFzaGnIpGD0QwYfnOSWQBAvkVXzWJ5zk1drpwzx2eqTqp7uJpU3UhhxbyK37ngqOIqCVN69OnVCXehZj7pLmWvMrzldBvPtT+u1wij6InxFbfC2dfxFMC3eGAGdHArteuLID7heOsbGla92ufr0z/w34X18ag3kELUqmr7IDOSbm7U6WooyNXjVQxIt3ThZ5/0oVX2bNz1ArLVsDXCv/imyL3LYEGWHJVPaNY7EQvVbXW0NSjCPf9G6x5v+SZUF6zaabFoBIyyvKs96z45A8=
2022-09-29T17:17:53.043 INFO:teuthology.kill:2022-09-29 17:17:53,041.041 INFO:teuthology.nuke:Checking targets against current locks
2022-09-29T17:17:53.106 INFO:teuthology.kill:2022-09-29 17:17:53,106.106 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-29T17:17:53.131 INFO:teuthology.kill:2022-09-29 17:17:53,131.131 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-29T17:17:53.191 INFO:teuthology.kill:2022-09-29 17:17:53,190.190 INFO:teuthology.orchestra.console:Power off smithi006
2022-09-29T17:17:53.237 INFO:teuthology.kill:2022-09-29 17:17:53,236.236 INFO:teuthology.orchestra.console:Power off smithi120
2022-09-29T17:18:01.577 INFO:teuthology.kill:2022-09-29 17:18:01,577.577 INFO:teuthology.orchestra.console:Power off for smithi006 completed
2022-09-29T17:18:01.601 INFO:teuthology.kill:2022-09-29 17:18:01,600.600 INFO:teuthology.orchestra.console:Power off for smithi120 completed
2022-09-29T17:18:01.744 INFO:teuthology.kill:2022-09-29 17:18:01,743.743 INFO:teuthology.lock.ops:unlocked: smithi006.front.sepia.ceph.com
2022-09-29T17:18:01.752 INFO:teuthology.kill:2022-09-29 17:18:01,751.751 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com
2022-09-29T17:20:02.150 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-09-29T17:20:02.331 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi006 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-29_16:40:30-rados-wip-all-kickoff-r-distro-default-smithi/7047746
2022-09-29T17:20:02.333 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi120 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-29_16:40:30-rados-wip-all-kickoff-r-distro-default-smithi/7047744