2022-03-04T16:22:07.514 INFO:root:teuthology version: 1.1.0-eeaeee38 2022-03-04T16:22:07.645 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging 2022-03-04T16:22:07.646 INFO:teuthology.lock.ops:Updating [ubuntu@smithi120.front.sepia.ceph.com]: reset os type and version on server 2022-03-04T16:22:07.647 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2022-03-04T16:22:07.665 INFO:teuthology.lock.ops:Node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging is complete 2022-03-04T16:22:07.665 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging 2022-03-04T16:22:07.666 INFO:teuthology.lock.ops:Updating [ubuntu@smithi155.front.sepia.ceph.com]: reset os type and version on server 2022-03-04T16:22:07.666 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2022-03-04T16:22:07.685 INFO:teuthology.lock.ops:Node 'ubuntu@smithi155.front.sepia.ceph.com' reimaging is complete 2022-03-04T16:22:07.882 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.4 2022-03-04T16:22:07.887 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of rhel 8.4 2022-03-04T16:22:08.204 INFO:teuthology.orchestra.console:Power off smithi155 2022-03-04T16:22:08.218 INFO:teuthology.orchestra.console:Power off smithi120 2022-03-04T16:22:16.568 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2022-03-04T16:22:16.578 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2022-03-04T16:22:16.670 INFO:teuthology.orchestra.console:Power on smithi155 2022-03-04T16:22:16.679 INFO:teuthology.orchestra.console:Power on smithi120 2022-03-04T16:22:25.037 INFO:teuthology.orchestra.console:Power on for smithi155 completed 2022-03-04T16:22:25.042 INFO:teuthology.orchestra.console:Power on for smithi120 completed 2022-03-04T16:22:25.139 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2022-03-04T16:22:25.143 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish 2022-03-04T16:27:03.032 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-04T16:27:03.236 INFO:teuthology.provision.fog.smithi120:Node is ready 2022-03-04T16:27:03.291 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com 2022-03-04T16:27:03.346 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120 2022-03-04T16:27:03.667 INFO:teuthology.provision.fog.smithi120:Deploy complete! 2022-03-04T16:27:09.010 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-04T16:27:09.244 INFO:teuthology.provision.fog.smithi155:Node is ready 2022-03-04T16:27:09.259 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2022-03-04T16:27:09.314 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2022-03-04T16:27:09.629 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2022-03-04T16:27:09.772 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com 2022-03-04T16:27:09.773 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-04T16:27:09.794 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2022-03-04T16:27:09.794 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-04T16:27:09.816 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server 2022-03-04T16:27:09.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2022-03-04T16:27:10.050 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2022-03-04T16:27:10.106 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux" 2022-03-04T16:27:10.107 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.4 (Ootpa)" 2022-03-04T16:27:10.107 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel" 2022-03-04T16:27:10.108 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora" 2022-03-04T16:27:10.108 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.4" 2022-03-04T16:27:10.108 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8" 2022-03-04T16:27:10.108 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-03-04T16:27:10.109 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2022-03-04T16:27:10.109 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-03-04T16:27:10.109 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/" 2022-03-04T16:27:10.109 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-03-04T16:27:10.109 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-04T16:27:10.110 INFO:teuthology.orchestra.run.smithi120.stdout: 2022-03-04T16:27:10.110 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-04T16:27:10.110 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-03-04T16:27:10.111 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-03-04T16:27:10.111 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-03-04T16:27:10.112 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2022-03-04T16:27:10.130 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2022-03-04T16:27:10.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2022-03-04T16:27:10.394 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2022-03-04T16:27:10.448 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux" 2022-03-04T16:27:10.449 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.4 (Ootpa)" 2022-03-04T16:27:10.449 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel" 2022-03-04T16:27:10.449 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora" 2022-03-04T16:27:10.449 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.4" 2022-03-04T16:27:10.449 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2022-03-04T16:27:10.450 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-03-04T16:27:10.450 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2022-03-04T16:27:10.450 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-03-04T16:27:10.450 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/" 2022-03-04T16:27:10.451 INFO:teuthology.orchestra.run.smithi155.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-03-04T16:27:10.451 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-04T16:27:10.451 INFO:teuthology.orchestra.run.smithi155.stdout: 2022-03-04T16:27:10.451 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-04T16:27:10.452 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-03-04T16:27:10.452 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-03-04T16:27:10.452 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-03-04T16:27:10.453 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2022-03-04T16:27:10.537 INFO:teuthology.dispatcher.supervisor:Running job 6720404 2022-03-04T16:27:10.537 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-04_16:10:03-rados-wip-yuri4-testing-2022-03-03-1448-distro-default-smithi/6720404 --name yuriw-2022-03-04_16:10:03-rados-wip-yuri4-testing-2022-03-03-1448-distro-default-smithi --description rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} -- /home/teuthworker/archive/yuriw-2022-03-04_16:10:03-rados-wip-yuri4-testing-2022-03-03-1448-distro-default-smithi/6720404/orig.config.yaml 2022-03-04T16:27:10.540 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-04_16:10:03-rados-wip-yuri4-testing-2022-03-03-1448-distro-default-smithi/6720404 2022-03-04T16:27:10.541 INFO:teuthology.dispatcher.supervisor:Job PID: 16120 2022-03-04T16:27:10.541 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-04T22:57:39.819 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-04T22:57:39.989 INFO:teuthology.kill:Killing Pids: {16120} 2022-03-04T22:57:41.203 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2022-03-04T22:57:41.204 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['osd.3', 'osd.4', 'osd.5', 'client.0'] 2022-03-04T22:57:41.204 INFO:teuthology.misc:Compressing logs... 2022-03-04T22:57:41.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2022-03-04T22:57:41.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2022-03-04T22:57:41.796 INFO:teuthology.misc:Compressing logs... 2022-03-04T22:57:42.254 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-03-04T22:57:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-03-04T22:57:42.664 INFO:teuthology.misc:Compressing logs... 2022-03-04T23:00:34.496 INFO:teuthology.kill:No teuthology processes running 2022-03-04T23:00:35.594 INFO:teuthology.kill:Nuking machines: ['smithi120', 'smithi155'] 2022-03-04T23:00:36.580 INFO:teuthology.kill:2022-03-04 23:00:36,580.580 INFO:teuthology.nuke:targets: 2022-03-04T23:00:36.581 INFO:teuthology.kill: smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDp3QCGcoSv6m7cdUfK/Cxmkjg75W4MHnHYyPBHQFMf/fRWINyca1rkabmweYOBFG20wC4V81kGhVPefhbabG8AeaxJVra0ctI+7rOz6OZyR9tNXuE2LiiO3HB6lntLGxk4+uze2+1f64dbTT4xUEW5TZlbt4T/H+NNDEZwgAvuj+a4EDOq3OlcipmKDYzHqU7Fae5QJh/mV0wcyEc1s8DsBHkHlSb6LiRjSuyoYUYmoasgJZChXWPnkaEJfPLYyPwPogVE/0aN4iEyR5C+n8aPuXvi0i1TWp03cvxt3F/1HRveLlwt2ISK7pHrad/TGN9dzJ1YGySubB8jPi6o61lZOTNnUwm+r0iVBtpfAjeJND/klPICuLvhmhG+O+XbsDEHVIckHQBY/AdIAK4OqmeoqqiCZeMsEhkzV/+0R2P+JskeM0We0cstWWrpBsbnUj3fpl7TgSjPTY343DwPm7/8q+Xrl6YsFllgitZCmD6Fx3cGnE2k508lYOaKLuSoQFk= 2022-03-04T23:00:36.582 INFO:teuthology.kill: smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9G9bs3+ySLHAjaUHZUlMYvPqYdKoUlcEE3Q4UmrRWDIK4oaud234l3yJMOqh+hY2ifwrI85AiANwbWJHtMVcNCYATrrF2r4UF7FuPsD2nTscqotci959M1fb8yERiXtewfqT52y7hibb7Lm9sZx3RZzSLp7AYIVZbf3/Rl7yVl8ED5kRRNnqN8ZB47u5hEf1l9wX3qxQ3M7YPfLwpf/hHt4Nd7k90Y1TO86D4Gv0a+w0Neqs4CWRR6eHqUwtz7UiCcX8qpMq8EPU7VJUMKxIbtTXzGzGVr6IiIvhrlUwlTZzk98JzBLP2wPUem4mqSteSuhC4uQAeW2PditTliQ5rjHYbtQNzwa5IYCVm84mqH+NhiepZd96VUETddkPl8VI3bhFzNu1BuEBchtaZSW0BSsel1DOHfhC6kIpZTnHAmplljaN91qsoh9oLIyI0skltkBmxr+7c6FPaV2cy7BcVV3OZKYpQIJCwvg9B999P4JLozapB3rQ194dK+hiVoeU= 2022-03-04T23:00:36.607 INFO:teuthology.kill:2022-03-04 23:00:36,607.607 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-04T23:00:36.610 INFO:teuthology.kill:2022-03-04 23:00:36,610.610 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-04T23:00:36.685 INFO:teuthology.kill:2022-03-04 23:00:36,685.685 INFO:teuthology.orchestra.console:Power off smithi155 2022-03-04T23:00:36.696 INFO:teuthology.kill:2022-03-04 23:00:36,696.696 INFO:teuthology.orchestra.console:Power off smithi120 2022-03-04T23:00:49.169 INFO:teuthology.kill:2022-03-04 23:00:49,168.168 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2022-03-04T23:00:49.174 INFO:teuthology.kill:2022-03-04 23:00:49,174.174 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2022-03-04T23:00:49.313 INFO:teuthology.kill:2022-03-04 23:00:49,313.313 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com 2022-03-04T23:00:49.317 INFO:teuthology.kill:2022-03-04 23:00:49,316.316 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com 2022-03-04T23:02:49.594 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-04T23:02:49.687 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi120 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-04_21:54:24-upgrade-wip-yuri4-testing-2022-03-03-1448-distro-default-smithi/6721177 2022-03-04T23:02:49.688 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi155 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-04_21:54:24-upgrade-wip-yuri4-testing-2022-03-03-1448-distro-default-smithi/6721177