2022-03-15T02:40:26.429 INFO:root:teuthology version: 1.1.0-fbfc7b38
2022-03-15T02:40:26.555 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi089.front.sepia.ceph.com' reimaging
2022-03-15T02:40:26.556 INFO:teuthology.lock.ops:Updating [ubuntu@smithi089.front.sepia.ceph.com]: reset os type and version on server
2022-03-15T02:40:26.557 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2022-03-15T02:40:26.576 INFO:teuthology.lock.ops:Node 'ubuntu@smithi089.front.sepia.ceph.com' reimaging is complete
2022-03-15T02:40:26.576 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi183.front.sepia.ceph.com' reimaging
2022-03-15T02:40:26.576 INFO:teuthology.lock.ops:Updating [ubuntu@smithi183.front.sepia.ceph.com]: reset os type and version on server
2022-03-15T02:40:26.577 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2022-03-15T02:40:26.595 INFO:teuthology.lock.ops:Node 'ubuntu@smithi183.front.sepia.ceph.com' reimaging is complete
2022-03-15T02:40:26.792 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of rhel 8.4
2022-03-15T02:40:26.826 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of rhel 8.4
2022-03-15T02:40:27.188 INFO:teuthology.orchestra.console:Power off smithi089
2022-03-15T02:40:27.203 INFO:teuthology.orchestra.console:Power off smithi183
2022-03-15T02:40:35.563 INFO:teuthology.orchestra.console:Power off for smithi089 completed
2022-03-15T02:40:35.568 INFO:teuthology.orchestra.console:Power off for smithi183 completed
2022-03-15T02:40:35.664 INFO:teuthology.orchestra.console:Power on smithi089
2022-03-15T02:40:35.675 INFO:teuthology.orchestra.console:Power on smithi183
2022-03-15T02:40:44.035 INFO:teuthology.orchestra.console:Power on for smithi089 completed
2022-03-15T02:40:44.042 INFO:teuthology.orchestra.console:Power on for smithi183 completed
2022-03-15T02:40:44.137 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish
2022-03-15T02:40:44.143 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish
2022-03-15T02:45:16.076 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-15T02:45:16.116 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-15T02:45:16.293 INFO:teuthology.provision.fog.smithi183:Node is ready
2022-03-15T02:45:16.308 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com
2022-03-15T02:45:16.328 INFO:teuthology.provision.fog.smithi089:Node is ready
2022-03-15T02:45:16.344 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com
2022-03-15T02:45:16.365 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183
2022-03-15T02:45:16.400 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089
2022-03-15T02:45:16.697 INFO:teuthology.provision.fog.smithi183:Deploy complete!
2022-03-15T02:45:16.726 INFO:teuthology.provision.fog.smithi089:Deploy complete!
2022-03-15T02:45:16.873 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com
2022-03-15T02:45:16.874 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-15T02:45:16.893 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com
2022-03-15T02:45:16.894 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-15T02:45:16.914 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server
2022-03-15T02:45:16.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com'
2022-03-15T02:45:17.137 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64
2022-03-15T02:45:17.191 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Red Hat Enterprise Linux"
2022-03-15T02:45:17.192 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8.4 (Ootpa)"
2022-03-15T02:45:17.192 INFO:teuthology.orchestra.run.smithi089.stdout:ID="rhel"
2022-03-15T02:45:17.192 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="fedora"
2022-03-15T02:45:17.192 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8.4"
2022-03-15T02:45:17.192 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8"
2022-03-15T02:45:17.193 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-15T02:45:17.193 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31"
2022-03-15T02:45:17.194 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-15T02:45:17.194 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.redhat.com/"
2022-03-15T02:45:17.194 INFO:teuthology.orchestra.run.smithi089.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-15T02:45:17.194 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-15T02:45:17.195 INFO:teuthology.orchestra.run.smithi089.stdout:
2022-03-15T02:45:17.195 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-15T02:45:17.195 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-15T02:45:17.195 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-15T02:45:17.196 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-15T02:45:17.197 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2022-03-15T02:45:17.227 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server
2022-03-15T02:45:17.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com'
2022-03-15T02:45:17.462 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64
2022-03-15T02:45:17.516 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Red Hat Enterprise Linux"
2022-03-15T02:45:17.517 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8.4 (Ootpa)"
2022-03-15T02:45:17.517 INFO:teuthology.orchestra.run.smithi183.stdout:ID="rhel"
2022-03-15T02:45:17.517 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="fedora"
2022-03-15T02:45:17.518 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8.4"
2022-03-15T02:45:17.518 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8"
2022-03-15T02:45:17.518 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-03-15T02:45:17.518 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31"
2022-03-15T02:45:17.519 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-03-15T02:45:17.519 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.redhat.com/"
2022-03-15T02:45:17.519 INFO:teuthology.orchestra.run.smithi183.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-03-15T02:45:17.519 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-15T02:45:17.519 INFO:teuthology.orchestra.run.smithi183.stdout:
2022-03-15T02:45:17.520 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-15T02:45:17.520 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-03-15T02:45:17.520 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-03-15T02:45:17.520 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-03-15T02:45:17.521 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2022-03-15T02:45:17.629 INFO:teuthology.dispatcher.supervisor:Running job 6736891
2022-03-15T02:45:17.629 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-14_18:57:01-fs-wip-yuri2-testing-2022-03-14-0946-quincy-distro-default-smithi/6736891 --name yuriw-2022-03-14_18:57:01-fs-wip-yuri2-testing-2022-03-14-0946-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} -- /home/teuthworker/archive/yuriw-2022-03-14_18:57:01-fs-wip-yuri2-testing-2022-03-14-0946-quincy-distro-default-smithi/6736891/orig.config.yaml
2022-03-15T02:45:17.632 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-14_18:57:01-fs-wip-yuri2-testing-2022-03-14-0946-quincy-distro-default-smithi/6736891
2022-03-15T02:45:17.633 INFO:teuthology.dispatcher.supervisor:Job PID: 24640
2022-03-15T02:45:17.634 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-15T09:15:47.281 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-15T09:15:47.550 INFO:teuthology.kill:Killing Pids: {24640}
2022-03-15T09:15:48.689 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2']
2022-03-15T09:15:48.690 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']
2022-03-15T09:15:48.690 INFO:teuthology.misc:Compressing logs...
2022-03-15T09:15:48.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com'
2022-03-15T09:15:48.881 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com'
2022-03-15T09:15:49.209 INFO:teuthology.misc:Compressing logs...
2022-03-15T09:15:49.366 INFO:teuthology.misc:Compressing logs...
2022-03-15T09:16:58.046 INFO:teuthology.kill:No teuthology processes running
2022-03-15T09:16:59.189 INFO:teuthology.kill:Nuking machines: ['smithi089', 'smithi183']
2022-03-15T09:17:00.119 INFO:teuthology.kill:2022-03-15 09:17:00,119.119 INFO:teuthology.nuke:targets:
2022-03-15T09:17:00.120 INFO:teuthology.kill:  smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0xnCF7rn/Gouf0Es0ca+QOWDQOlE+IdPm/oHi6cxjm17Rr2O50QwqQHjUjsM1VnE1fK4/OXaA01kEBUEI6EZ6zajtfm/51zDyaylEBA1P/YwH6pL1C511+6GDogYpJwOqICh8vfter+JI4IKTy36uyIblcoxxTWPHdjfHRrHdRf/ZNH1zlZN62g32JqvaFekUbslCSjMy7fxddfePHuBTB95Puj8m9GxiNc1VowUBSJz75SghpRVefm1debcFtayU0S/MqsA2UBg76QzUNSy8OwR0HmhY91A7XrH3AnQ6h2aotiaW4yomwW8IYI6CjdWjTnXPKdFn5Bd/3xggS+/5FvQz1oRXM8MIuPUfQMyv5iufIyoNpQs/fDsosvVAwQkYoeOrr1sJb2oBd/1Ja/UYGL1nxFwJRb/MWOqLDZTMpv+QWeuxFNCn6WqdHJtxSLx7gxePv8FIqIbJ5ctWdXGa/14whqfHh/wekKrTcrZoF87BCidLRJDHtjp5sIue4qk=
2022-03-15T09:17:00.120 INFO:teuthology.kill:  smithi183.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfRgPgBxRrba0JOW7IfC5iC9lyDJTsaSSXXNU1k1UO+LzXK6bZOmD/R73utjf5MfmC21uIt+siqmAs/i9Ozl/1M3JGTQ5DzpPItPofH6YURGkn1/htIk+33vlO2uNL+fyrB5VUd7ey24D4z/pkm/sX5LFCAsE9UHfo5Dgu426ai1erjWKUMzqepjakfIWLyYDUchUE5T3KO1Y87gsFJVuMJlCV7YixAYVfr56ueb0Ove3b1B1oROqfM+81UQO0iIdrnJqYkQp5F0hXmUT9QwhVEAUxaa4RHffOvn7I4+4a2KqkoJOmCqdm14FR3KKicXWdVJ1HPNS1yKLNSLUBmY3JXCY7qh1h4+3fsOGn8jyh3JqsO3wAAAWWaP68nxa0fQXjkq8j+W/jsoZrxNj80rjBgTfbyYy37B6qnpF3fHGkHEfGTHIGi9pnxr9iMJ5u1QkvWhPFkXoNNmLlev1zEknZ2W+WC/79NUJjk0l6VlcqyW7GjedF5r9Fc28Lr97XQa8=
2022-03-15T09:17:00.146 INFO:teuthology.kill:2022-03-15 09:17:00,146.146 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-15T09:17:00.150 INFO:teuthology.kill:2022-03-15 09:17:00,150.150 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-15T09:17:00.238 INFO:teuthology.kill:2022-03-15 09:17:00,238.238 INFO:teuthology.orchestra.console:Power off smithi183
2022-03-15T09:17:00.251 INFO:teuthology.kill:2022-03-15 09:17:00,251.251 INFO:teuthology.orchestra.console:Power off smithi089
2022-03-15T09:17:08.612 INFO:teuthology.kill:2022-03-15 09:17:08,611.611 INFO:teuthology.orchestra.console:Power off for smithi183 completed
2022-03-15T09:17:08.617 INFO:teuthology.kill:2022-03-15 09:17:08,617.617 INFO:teuthology.orchestra.console:Power off for smithi089 completed
2022-03-15T09:17:08.762 INFO:teuthology.kill:2022-03-15 09:17:08,761.761 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com
2022-03-15T09:17:08.764 INFO:teuthology.kill:2022-03-15 09:17:08,764.764 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com
2022-03-15T09:19:09.094 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-15T09:19:09.204 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi089 but it was locked by another job: /home/teuthworker/archive/yuvalif-2022-03-15_08:46:16-rgw:verify-wip-arrow-submodule-ext-distro-basic-smithi/6737980
2022-03-15T09:19:09.205 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi183 but it was locked by another job: /home/teuthworker/archive/yuvalif-2022-03-15_08:46:16-rgw:verify-wip-arrow-submodule-ext-distro-basic-smithi/6737980