2022-10-25T21:57:27.555 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-25T21:57:27.718 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging 2022-10-25T21:57:27.719 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server 2022-10-25T21:57:27.720 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2022-10-25T21:57:27.748 INFO:teuthology.lock.ops:Node 'smithi153.front.sepia.ceph.com' reimaging is complete 2022-10-25T21:57:27.749 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging 2022-10-25T21:57:27.749 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server 2022-10-25T21:57:27.750 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2022-10-25T21:57:27.774 INFO:teuthology.lock.ops:Node 'smithi164.front.sepia.ceph.com' reimaging is complete 2022-10-25T21:57:27.976 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of rhel 8.4 2022-10-25T21:57:27.985 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of rhel 8.4 2022-10-25T21:57:28.320 INFO:teuthology.orchestra.console:Power off smithi153 2022-10-25T21:57:28.336 INFO:teuthology.orchestra.console:Power off smithi164 2022-10-25T21:57:28.467 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2022-10-25T21:57:28.475 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2022-10-25T21:57:28.569 INFO:teuthology.orchestra.console:Power on smithi153 2022-10-25T21:57:28.577 INFO:teuthology.orchestra.console:Power on smithi164 2022-10-25T21:57:32.823 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2022-10-25T21:57:32.924 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2022-10-25T21:57:36.942 INFO:teuthology.orchestra.console:Power on for smithi164 completed 2022-10-25T21:57:37.043 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2022-10-25T22:02:16.816 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-25T22:02:17.043 INFO:teuthology.provision.fog.smithi153:Node is ready 2022-10-25T22:02:17.059 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2022-10-25T22:02:17.114 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2022-10-25T22:02:17.447 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2022-10-25T22:02:39.042 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-25T22:02:39.275 INFO:teuthology.provision.fog.smithi164:Node is ready 2022-10-25T22:02:39.290 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2022-10-25T22:02:39.346 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2022-10-25T22:02:39.679 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2022-10-25T22:02:39.857 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2022-10-25T22:02:39.858 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-25T22:02:39.878 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2022-10-25T22:02:39.878 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-25T22:02:39.898 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2022-10-25T22:02:39.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2022-10-25T22:02:40.136 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2022-10-25T22:02:40.192 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Red Hat Enterprise Linux" 2022-10-25T22:02:40.193 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8.4 (Ootpa)" 2022-10-25T22:02:40.193 INFO:teuthology.orchestra.run.smithi153.stdout:ID="rhel" 2022-10-25T22:02:40.193 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="fedora" 2022-10-25T22:02:40.194 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8.4" 2022-10-25T22:02:40.194 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2022-10-25T22:02:40.194 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-10-25T22:02:40.194 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2022-10-25T22:02:40.195 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-10-25T22:02:40.195 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.redhat.com/" 2022-10-25T22:02:40.195 INFO:teuthology.orchestra.run.smithi153.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-10-25T22:02:40.195 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-25T22:02:40.195 INFO:teuthology.orchestra.run.smithi153.stdout: 2022-10-25T22:02:40.196 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-25T22:02:40.196 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-10-25T22:02:40.196 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-10-25T22:02:40.196 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-10-25T22:02:40.198 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2022-10-25T22:02:40.218 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2022-10-25T22:02:40.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-10-25T22:02:40.446 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2022-10-25T22:02:40.499 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Red Hat Enterprise Linux" 2022-10-25T22:02:40.500 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8.4 (Ootpa)" 2022-10-25T22:02:40.500 INFO:teuthology.orchestra.run.smithi164.stdout:ID="rhel" 2022-10-25T22:02:40.500 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="fedora" 2022-10-25T22:02:40.501 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8.4" 2022-10-25T22:02:40.501 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2022-10-25T22:02:40.501 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-10-25T22:02:40.501 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2022-10-25T22:02:40.502 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-10-25T22:02:40.502 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.redhat.com/" 2022-10-25T22:02:40.502 INFO:teuthology.orchestra.run.smithi164.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-10-25T22:02:40.502 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-25T22:02:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-10-25T22:02:40.503 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-25T22:02:40.503 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-10-25T22:02:40.503 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-10-25T22:02:40.503 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-10-25T22:02:40.504 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2022-10-25T22:02:40.608 INFO:teuthology.dispatcher.supervisor:Running job 7080258 2022-10-25T22:02:40.608 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080258 --name teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} -- /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080258/orig.config.yaml 2022-10-25T22:02:40.613 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080258 2022-10-25T22:02:40.614 INFO:teuthology.dispatcher.supervisor:Job PID: 32484 2022-10-25T22:02:40.615 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-10-26T10:03:27.489 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-10-26T10:03:28.062 INFO:teuthology.kill:Killing Pids: {32484} 2022-10-26T10:03:29.348 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-10-26T10:03:29.348 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-10-26T10:03:29.349 INFO:teuthology.misc:Compressing logs... 2022-10-26T10:03:29.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2022-10-26T10:03:29.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-10-26T10:03:29.988 INFO:teuthology.kill:No teuthology processes running 2022-10-26T10:03:31.146 INFO:teuthology.kill:Nuking machines: ['smithi153', 'smithi164'] 2022-10-26T10:03:32.149 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-10-26T10:03:32.150 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-10-26T10:03:32.151 INFO:teuthology.kill:2022-10-26 10:03:32,149.149 INFO:teuthology.nuke:targets: 2022-10-26T10:03:32.151 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZwObp6Yh8yOelG3xUgMnLg47OUzEWvLH6tN/RghnlsjQnqX0AsghdBezQVTVNYZKj9jF16AeCZi696+2p8UDlgKk0K97GtVDuZEB+8dCMZq1xvIBnfF1VcrTAidqhdTBDtrc1YkOaj8+CTnD5jQvQkbinJrbyXIAQiDNMG+q5xUaw5Be29W0u6nTioOlAMvTXhED+uplq4kUweseAegdurpbc6hJNzJQPkb1NJGj+nscpgDpgdCWkzI6y++YmtNEBIV8DhatAgP6Nd6kmHjn8mZxs41vjjxjpU0kiRoiO4O71XboAaMvjGLnlXApiVUy73LUBDwvuwlL2lOSHsOokWa7tlPfjXHPzQjN3dUBo9ISJ5ID+sp2lKBKIuFnEt3dHZNJbVUUVgEvSBT6Cbf70Yz6MoAaA3v25fOH3chztINL4tcPruXV5LZazArH0RWlDt55/6BvZiKw7dbuDXgAMsRwvsjLIYy/PUKg0NJWmFp6Z01eL74F1dWr0GT0nj4k= 2022-10-26T10:03:32.151 INFO:teuthology.kill: smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqh8loBfxU7jeJAwL4gHLJdWfaWy90PpcK528C/DjPuLJ2HeCqGaTCDmoRkmCUbTHQ6oZNYiPoSzcDYF9rD3pJy0ymwshBuqcSlABvvecs7c3z4MdveFnrrioM+t9omOnjKVOVXvoURzwVno+5YOwYHDd//hAd9aazC1eCBicgVHrOdYiWC+5cUjqBpsNf17YYOyEDOspccyUtCngUWo5gVu5bh7P4G0+XI1967liQkVZFGrgq2R1ovp64HUC9EZWShjkVaGZx/xiGxHe5vsUYOxL0avJ9vQJDGj4i2o7oy1vZdo94uJiNm0ZtvHbWmLaJ/LAFbBDo1Wtd+juqYCEBuTTQTKF6AiR8KtvkwaYXeIghRG6/b8xediiXH7jJtfcsL/hR4riB6BsdjFkR6Q2h7L591ltT2GvNYmm11jfsZ+eYOPaqqp4hPNyCFOWhTlsQn1SrmJz+gKFLxDUscNW8XcnSzTW8qV6ZkeAUuveXLDLWQ0cDq7eHZncgbGrJrfU= 2022-10-26T10:03:32.151 INFO:teuthology.kill:2022-10-26 10:03:32,149.149 INFO:teuthology.nuke:Checking targets against current locks 2022-10-26T10:03:32.199 INFO:teuthology.kill:2022-10-26 10:03:32,199.199 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-26T10:03:32.231 INFO:teuthology.kill:2022-10-26 10:03:32,231.231 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-26T10:03:32.275 INFO:teuthology.kill:2022-10-26 10:03:32,275.275 INFO:teuthology.orchestra.console:Power off smithi153 2022-10-26T10:03:32.313 INFO:teuthology.kill:2022-10-26 10:03:32,312.312 INFO:teuthology.orchestra.console:Power off smithi164 2022-10-26T10:03:40.648 INFO:teuthology.kill:2022-10-26 10:03:40,648.648 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2022-10-26T10:03:40.796 INFO:teuthology.kill:2022-10-26 10:03:40,796.796 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com 2022-10-26T10:03:44.784 INFO:teuthology.kill:2022-10-26 10:03:44,783.783 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2022-10-26T10:03:44.934 INFO:teuthology.kill:2022-10-26 10:03:44,933.933 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com 2022-10-26T10:05:45.162 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-10-26T10:05:45.271 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/dis-2022-10-25_23:10:37-rbd-wip-dis-testing-distro-default-smithi/7080896 2022-10-26T10:05:45.272 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi164 but it was locked by another job: /home/teuthworker/archive/dis-2022-10-25_23:10:37-rbd-wip-dis-testing-distro-default-smithi/7080896