2022-02-18T14:39:29.547 INFO:root:teuthology version: 1.1.0-eea25212
2022-02-18T14:39:29.675 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi109.front.sepia.ceph.com' reimaging
2022-02-18T14:39:29.676 INFO:teuthology.lock.ops:Updating [ubuntu@smithi109.front.sepia.ceph.com]: reset os type and version on server
2022-02-18T14:39:29.676 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server
2022-02-18T14:39:29.697 INFO:teuthology.lock.ops:Node 'ubuntu@smithi109.front.sepia.ceph.com' reimaging is complete
2022-02-18T14:39:29.699 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi135.front.sepia.ceph.com' reimaging
2022-02-18T14:39:29.699 INFO:teuthology.lock.ops:Updating [ubuntu@smithi135.front.sepia.ceph.com]: reset os type and version on server
2022-02-18T14:39:29.699 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server
2022-02-18T14:39:29.717 INFO:teuthology.lock.ops:Node 'ubuntu@smithi135.front.sepia.ceph.com' reimaging is complete
2022-02-18T14:39:29.913 INFO:teuthology.provision.fog.smithi109:Scheduling deploy of rhel 8.4
2022-02-18T14:39:29.921 INFO:teuthology.provision.fog.smithi135:Scheduling deploy of rhel 8.4
2022-02-18T14:39:30.204 INFO:teuthology.orchestra.console:Power off smithi109
2022-02-18T14:39:30.241 INFO:teuthology.orchestra.console:Power off smithi135
2022-02-18T14:39:42.691 INFO:teuthology.orchestra.console:Power off for smithi109 completed
2022-02-18T14:39:42.726 INFO:teuthology.orchestra.console:Power off for smithi135 completed
2022-02-18T14:39:42.793 INFO:teuthology.orchestra.console:Power on smithi109
2022-02-18T14:39:42.827 INFO:teuthology.orchestra.console:Power on smithi135
2022-02-18T14:39:47.080 INFO:teuthology.orchestra.console:Power on for smithi135 completed
2022-02-18T14:39:47.181 INFO:teuthology.provision.fog.smithi135:Waiting for deploy to finish
2022-02-18T14:39:51.165 INFO:teuthology.orchestra.console:Power on for smithi109 completed
2022-02-18T14:39:51.266 INFO:teuthology.provision.fog.smithi109:Waiting for deploy to finish
2022-02-18T14:44:31.215 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-18T14:44:31.436 INFO:teuthology.provision.fog.smithi135:Node is ready
2022-02-18T14:44:31.452 INFO:teuthology.orchestra.run.smithi135.stdout:smithi135.front.sepia.ceph.com
2022-02-18T14:44:31.507 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135 smithi135.front.sepia.ceph.com smithi135
2022-02-18T14:44:31.840 INFO:teuthology.provision.fog.smithi135:Deploy complete!
2022-02-18T14:44:35.014 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-18T14:44:35.236 INFO:teuthology.provision.fog.smithi109:Node is ready
2022-02-18T14:44:35.251 INFO:teuthology.orchestra.run.smithi109.stdout:smithi109.front.sepia.ceph.com
2022-02-18T14:44:35.306 INFO:teuthology.orchestra.run.smithi109.stdout:172.21.15.109 smithi109.front.sepia.ceph.com smithi109
2022-02-18T14:44:35.634 INFO:teuthology.provision.fog.smithi109:Deploy complete!
2022-02-18T14:44:35.783 INFO:teuthology.lock.ops:Checking smithi109.front.sepia.ceph.com
2022-02-18T14:44:35.784 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-18T14:44:35.807 INFO:teuthology.lock.ops:Checking smithi135.front.sepia.ceph.com
2022-02-18T14:44:35.807 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-18T14:44:35.827 INFO:teuthology.lock.ops:Updating [smithi109.front.sepia.ceph.com]: set os type and version on server
2022-02-18T14:44:35.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com'
2022-02-18T14:44:36.054 INFO:teuthology.orchestra.run.smithi109.stdout:x86_64
2022-02-18T14:44:36.108 INFO:teuthology.orchestra.run.smithi109.stdout:NAME="Red Hat Enterprise Linux"
2022-02-18T14:44:36.109 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION="8.4 (Ootpa)"
2022-02-18T14:44:36.109 INFO:teuthology.orchestra.run.smithi109.stdout:ID="rhel"
2022-02-18T14:44:36.109 INFO:teuthology.orchestra.run.smithi109.stdout:ID_LIKE="fedora"
2022-02-18T14:44:36.109 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION_ID="8.4"
2022-02-18T14:44:36.110 INFO:teuthology.orchestra.run.smithi109.stdout:PLATFORM_ID="platform:el8"
2022-02-18T14:44:36.110 INFO:teuthology.orchestra.run.smithi109.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-02-18T14:44:36.110 INFO:teuthology.orchestra.run.smithi109.stdout:ANSI_COLOR="0;31"
2022-02-18T14:44:36.110 INFO:teuthology.orchestra.run.smithi109.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-02-18T14:44:36.111 INFO:teuthology.orchestra.run.smithi109.stdout:HOME_URL="https://www.redhat.com/"
2022-02-18T14:44:36.111 INFO:teuthology.orchestra.run.smithi109.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-02-18T14:44:36.111 INFO:teuthology.orchestra.run.smithi109.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-18T14:44:36.111 INFO:teuthology.orchestra.run.smithi109.stdout:
2022-02-18T14:44:36.112 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-18T14:44:36.112 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-02-18T14:44:36.112 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-02-18T14:44:36.112 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-02-18T14:44:36.114 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server
2022-02-18T14:44:36.131 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: set os type and version on server
2022-02-18T14:44:36.132 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com'
2022-02-18T14:44:36.369 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64
2022-02-18T14:44:36.424 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Red Hat Enterprise Linux"
2022-02-18T14:44:36.425 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="8.4 (Ootpa)"
2022-02-18T14:44:36.425 INFO:teuthology.orchestra.run.smithi135.stdout:ID="rhel"
2022-02-18T14:44:36.425 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="fedora"
2022-02-18T14:44:36.425 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="8.4"
2022-02-18T14:44:36.426 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el8"
2022-02-18T14:44:36.426 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-02-18T14:44:36.426 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31"
2022-02-18T14:44:36.426 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-02-18T14:44:36.427 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.redhat.com/"
2022-02-18T14:44:36.427 INFO:teuthology.orchestra.run.smithi135.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-02-18T14:44:36.427 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-18T14:44:36.427 INFO:teuthology.orchestra.run.smithi135.stdout:
2022-02-18T14:44:36.428 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-18T14:44:36.428 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-02-18T14:44:36.428 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-02-18T14:44:36.428 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-02-18T14:44:36.429 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server
2022-02-18T14:44:36.673 INFO:teuthology.dispatcher.supervisor:Running job 6694201
2022-02-18T14:44:36.674 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-02-18_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi/6694201 --name teuthology-2022-02-18_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi --description upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} -- /home/teuthworker/archive/teuthology-2022-02-18_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi/6694201/orig.config.yaml
2022-02-18T14:44:36.677 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-02-18_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi/6694201
2022-02-18T14:44:36.678 INFO:teuthology.dispatcher.supervisor:Job PID: 30197
2022-02-18T14:44:36.678 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-18T21:15:05.300 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-18T21:15:05.591 INFO:teuthology.kill:Killing Pids: {30197}
2022-02-18T21:15:06.733 INFO:teuthology.task.internal:roles: ubuntu@smithi109.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2022-02-18T21:15:06.734 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2022-02-18T21:15:06.735 INFO:teuthology.misc:Compressing logs...
2022-02-18T21:15:06.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com'
2022-02-18T21:15:06.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com'
2022-02-18T21:15:07.339 INFO:teuthology.misc:Compressing logs...
2022-02-18T21:15:07.456 INFO:teuthology.misc:Compressing logs...
2022-02-18T21:15:16.029 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-18T21:15:17.340 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-18T21:15:20.004 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2022-02-18T21:15:29.875 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2022-02-18T21:15:29.896 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-18T21:15:34.075 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2022-02-18T21:15:39.856 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-18T21:15:44.066 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-18T21:15:50.844 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-18T21:15:54.024 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-18T21:16:01.218 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-18T21:16:05.047 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/14a45b6e-90cc-11ec-8c35-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2022-02-18T21:16:26.489 INFO:teuthology.kill:No teuthology processes running
2022-02-18T21:16:27.602 INFO:teuthology.kill:Nuking machines: ['smithi109', 'smithi135']
2022-02-18T21:16:28.518 INFO:teuthology.kill:2022-02-18 21:16:28,518.518 INFO:teuthology.nuke:targets:
2022-02-18T21:16:28.519 INFO:teuthology.kill:  smithi109.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0hwYqpiXJN8vw+UWbmBQPmurGlE3sYgCapCEDcMt0gHFoVFpl+mNY5XZV1IRZG/Fo5rIqi0QyM1WTWlSa+5EdBtEqCCXjvZ9l8RJ337xNs27mBJZto81yNTiGkjME0PBBZlpFbfMmxTGc63wVYPN3Ab5fRlfKFw0IYS7em8UevUvnwXuDBFGqqgQIjNI2fnEreqj9DJ9RCn+CoIAMbNGqNQpzDhDxFRkl1JG83ifkz45jJEIB0M4WWp6ZDy9fM67D9gIATuIwzfIyAO5u1gFt7K7de9Oxpfrd9tXdVyrJJABEd52JKJXQLRAvIWUBW1XERHmIPRFit5bbQF1TNYzVirOXQJvaz7CuiAP8RW91gcuf8XLvT6H3eKy6oRwWmXcCV61SpC/GOBZa+Huohdcf35K1cXREFKpZCRuDBpxxHvDBlnXXMnYSzs7tbtElSqmGYrdqr3mhlyjW7FY2UFCgKaKGnQYjFTuBF+Vk2N2M+nhlKRf/n+wHotlDxKXrvWk=
2022-02-18T21:16:28.519 INFO:teuthology.kill:  smithi135.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEQr8mZXcg54ypWnvsFTL4vMlXDuUO2eKLF7nDmJ9SNRceoXANBOd8RFFv+RzaJtsPliejS3X5W/Xl3lmLJrKeyi+y2735CUCBLVQJQaiPLGQ+fRnITDMYhfpZGl8WbcG6E7x20xYWLhkojtObnP/zffyjY6NgFhhNKFazAKjBbDENdtTHjnPlNf3BK+N1spuV7vZxxykNtRCvMv9pp1Pzq9n/QcAlNZiDdxIYyNVz0s00gt+I4GAJPvyxBnBtGO/Yy0XWKPQCkvM/ipxueGAzwAmo7s3sh6ooFX2KyHb7k5LmCXeG4ZbYaSr2MjqSagaW9HVBoPYCtHOTJSJmiigjPuW2JFR0xJeHNywZPqgASzFSX3HE4bZh0YIlhCPZBMmMpPbQpqTWjcQFxwN73kM1wtwFfXeOYDMYeRZ7t3agx4RwJD5R3YTjkqZJ4Yp8NTn1p9b8MVRkrvg0oC7tAfAXIKbG0o3RPZcFIHPtS0Sbh0lUP6NghhCbiFRulS0yEE0=
2022-02-18T21:16:28.544 INFO:teuthology.kill:2022-02-18 21:16:28,544.544 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-18T21:16:28.548 INFO:teuthology.kill:2022-02-18 21:16:28,547.547 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-18T21:16:28.622 INFO:teuthology.kill:2022-02-18 21:16:28,622.622 INFO:teuthology.orchestra.console:Power off smithi135
2022-02-18T21:16:28.635 INFO:teuthology.kill:2022-02-18 21:16:28,635.635 INFO:teuthology.orchestra.console:Power off smithi109
2022-02-18T21:16:41.106 INFO:teuthology.kill:2022-02-18 21:16:41,105.105 INFO:teuthology.orchestra.console:Power off for smithi135 completed
2022-02-18T21:16:41.111 INFO:teuthology.kill:2022-02-18 21:16:41,111.111 INFO:teuthology.orchestra.console:Power off for smithi109 completed
2022-02-18T21:16:41.257 INFO:teuthology.kill:2022-02-18 21:16:41,256.256 INFO:teuthology.lock.ops:unlocked: smithi135.front.sepia.ceph.com
2022-02-18T21:16:41.261 INFO:teuthology.kill:2022-02-18 21:16:41,261.261 INFO:teuthology.lock.ops:unlocked: smithi109.front.sepia.ceph.com
2022-02-18T21:18:41.549 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-18T21:18:41.675 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi109 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-18_20:48:16-upgrade-master-distro-default-smithi/6694435