2022-08-15T06:57:04.441 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc 2022-08-15T06:57:04.586 INFO:teuthology.lock.ops:Start node 'smithi083.front.sepia.ceph.com' reimaging 2022-08-15T06:57:04.587 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: reset os type and version on server 2022-08-15T06:57:04.587 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2022-08-15T06:57:04.608 INFO:teuthology.lock.ops:Node 'smithi083.front.sepia.ceph.com' reimaging is complete 2022-08-15T06:57:04.608 INFO:teuthology.lock.ops:Start node 'smithi109.front.sepia.ceph.com' reimaging 2022-08-15T06:57:04.608 INFO:teuthology.lock.ops:Updating [smithi109.front.sepia.ceph.com]: reset os type and version on server 2022-08-15T06:57:04.609 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server 2022-08-15T06:57:04.628 INFO:teuthology.lock.ops:Node 'smithi109.front.sepia.ceph.com' reimaging is complete 2022-08-15T06:57:04.825 INFO:teuthology.provision.fog.smithi109:Scheduling deploy of centos 8.stream 2022-08-15T06:57:04.829 INFO:teuthology.provision.fog.smithi083:Scheduling deploy of centos 8.stream 2022-08-15T06:57:05.125 INFO:teuthology.orchestra.console:Power off smithi083 2022-08-15T06:57:05.144 INFO:teuthology.orchestra.console:Power off smithi109 2022-08-15T06:57:13.512 INFO:teuthology.orchestra.console:Power off for smithi083 completed 2022-08-15T06:57:13.613 INFO:teuthology.orchestra.console:Power on smithi083 2022-08-15T06:57:17.628 INFO:teuthology.orchestra.console:Power off for smithi109 completed 2022-08-15T06:57:17.729 INFO:teuthology.orchestra.console:Power on smithi109 2022-08-15T06:57:22.001 INFO:teuthology.orchestra.console:Power on for smithi083 completed 2022-08-15T06:57:22.103 INFO:teuthology.provision.fog.smithi083:Waiting for deploy to finish 2022-08-15T06:57:26.093 INFO:teuthology.orchestra.console:Power on for smithi109 completed 2022-08-15T06:57:26.195 INFO:teuthology.provision.fog.smithi109:Waiting for deploy to finish 2022-08-15T06:59:53.239 ERROR:teuthology.orchestra.connection:Error authenticating with smithi083.front.sepia.ceph.com: Authentication failed. 2022-08-15T06:59:57.251 ERROR:teuthology.orchestra.connection:Error authenticating with smithi109.front.sepia.ceph.com: Authentication failed. 2022-08-15T07:01:38.277 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-08-15T07:01:58.262 INFO:teuthology.provision.fog.smithi109:Node is ready 2022-08-15T07:01:58.284 INFO:teuthology.orchestra.run.smithi109.stdout:smithi109.front.sepia.ceph.com 2022-08-15T07:01:58.343 INFO:teuthology.orchestra.run.smithi109.stdout:172.21.15.109 smithi109.front.sepia.ceph.com smithi109 2022-08-15T07:01:58.710 INFO:teuthology.provision.fog.smithi109:Deploy complete! 2022-08-15T07:02:06.567 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-08-15T07:02:06.862 INFO:teuthology.provision.fog.smithi083:Node is ready 2022-08-15T07:02:06.880 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083.front.sepia.ceph.com 2022-08-15T07:02:06.939 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83 smithi083.front.sepia.ceph.com smithi083 2022-08-15T07:02:07.288 INFO:teuthology.provision.fog.smithi083:Deploy complete! 2022-08-15T07:02:07.528 INFO:teuthology.lock.ops:Checking smithi083.front.sepia.ceph.com 2022-08-15T07:02:07.529 INFO:teuthology.lock.ops:New key found. Updating... 2022-08-15T07:02:07.553 INFO:teuthology.lock.ops:Checking smithi109.front.sepia.ceph.com 2022-08-15T07:02:07.554 INFO:teuthology.lock.ops:New key found. Updating... 2022-08-15T07:02:07.575 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: set os type and version on server 2022-08-15T07:02:07.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2022-08-15T07:02:07.731 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2022-08-15T07:02:07.789 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2022-08-15T07:02:07.789 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="8" 2022-08-15T07:02:07.789 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2022-08-15T07:02:07.789 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2022-08-15T07:02:07.790 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="8" 2022-08-15T07:02:07.790 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el8" 2022-08-15T07:02:07.790 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 8" 2022-08-15T07:02:07.790 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2022-08-15T07:02:07.791 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-08-15T07:02:07.791 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2022-08-15T07:02:07.791 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-08-15T07:02:07.791 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-08-15T07:02:07.792 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-08-15T07:02:07.793 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2022-08-15T07:02:07.813 INFO:teuthology.lock.ops:Updating [smithi109.front.sepia.ceph.com]: set os type and version on server 2022-08-15T07:02:07.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com' 2022-08-15T07:02:08.020 INFO:teuthology.orchestra.run.smithi109.stdout:x86_64 2022-08-15T07:02:08.079 INFO:teuthology.orchestra.run.smithi109.stdout:NAME="CentOS Stream" 2022-08-15T07:02:08.079 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION="8" 2022-08-15T07:02:08.080 INFO:teuthology.orchestra.run.smithi109.stdout:ID="centos" 2022-08-15T07:02:08.080 INFO:teuthology.orchestra.run.smithi109.stdout:ID_LIKE="rhel fedora" 2022-08-15T07:02:08.080 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION_ID="8" 2022-08-15T07:02:08.080 INFO:teuthology.orchestra.run.smithi109.stdout:PLATFORM_ID="platform:el8" 2022-08-15T07:02:08.081 INFO:teuthology.orchestra.run.smithi109.stdout:PRETTY_NAME="CentOS Stream 8" 2022-08-15T07:02:08.081 INFO:teuthology.orchestra.run.smithi109.stdout:ANSI_COLOR="0;31" 2022-08-15T07:02:08.081 INFO:teuthology.orchestra.run.smithi109.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-08-15T07:02:08.081 INFO:teuthology.orchestra.run.smithi109.stdout:HOME_URL="https://centos.org/" 2022-08-15T07:02:08.081 INFO:teuthology.orchestra.run.smithi109.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-08-15T07:02:08.082 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-08-15T07:02:08.082 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-08-15T07:02:08.082 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server 2022-08-15T07:02:08.198 INFO:teuthology.dispatcher.supervisor:Running job 6969461 2022-08-15T07:02:08.198 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_9e7483cc68a9eb6b54dacbb0bec3bf23a5d32425/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/6969461 --name vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/pjd}} -- /home/teuthworker/archive/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/6969461/orig.config.yaml 2022-08-15T07:02:08.203 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/6969461 2022-08-15T07:02:08.203 INFO:teuthology.dispatcher.supervisor:Job PID: 9178 2022-08-15T07:02:08.204 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-08-15T19:02:30.862 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-08-15T19:02:31.073 INFO:teuthology.kill:Killing Pids: {9178} 2022-08-15T19:02:32.588 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-08-15T19:02:32.590 INFO:teuthology.task.internal:roles: ubuntu@smithi109.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-08-15T19:02:32.590 INFO:teuthology.misc:Compressing logs... 2022-08-15T19:02:32.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2022-08-15T19:02:32.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com' 2022-08-15T19:02:33.688 INFO:teuthology.misc:Compressing logs... 2022-08-15T19:03:22.285 INFO:teuthology.misc:Compressing logs... 2022-08-15T19:06:53.628 INFO:teuthology.kill:No teuthology processes running 2022-08-15T19:06:54.985 INFO:teuthology.kill:Nuking machines: ['smithi083', 'smithi109'] 2022-08-15T19:06:56.128 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_9e7483cc68a9eb6b54dacbb0bec3bf23a5d32425/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-08-15T19:06:56.129 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-08-15T19:06:56.129 INFO:teuthology.kill:2022-08-15 19:06:56,127.127 INFO:teuthology.nuke:targets: 2022-08-15T19:06:56.129 INFO:teuthology.kill: smithi083.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxTIR0xfauEt15+/7uJcgyWJbAtHC61SsMxzRvvsU5wrXauty7oEL0/2lyC0WzpAv7W2Ai2ynrJnZKa8Kf4x/IgfWeL2YbmsjI5O13U6El2p/HeO/a1Q5GbzjqQnAtnceEUVstVrx6vXA9sVLLfblB5gl3FzN5Z8S6+GLFlpt4OmSrZNavxcCIfpWC2ufXdkin0ysyr22Bv2HDDPqobXCWsvhujcxRXMNzpltoSJoDbeLXFeS2PvlIZmHMKL9NZtiuSjfyKNSXoaGqdKdRRtVn98XbbLqnr0Uo/gI07FZioMtNrKtOda3+Ud06Z+1EtbFtRYg6MWi7TGgI/qOUwVtDOAN6c+Irwty1sWsaru6BZ2zYuUC3+i5A9LOS4lun3XU7ItWHCAea57LHTW+eXEI9ODt4nI8LS50PpcsMWFeDr45SWFyzS6Dd4+TmWWlXFlYnhhQPD7ppe0hiH/XEI2W0GxX/wG1c3zNtM+U6XN2EAZfw5jiUEdG66L4TntIDU8k= 2022-08-15T19:06:56.130 INFO:teuthology.kill: smithi109.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdz5uUB9fevFTuxw1KmktNqmqaDQQcvlCP85c0IoZdAMJj3ADBOk99W8NF6lEfsDBcV4fAw1oDjFk4Rp/aaPW+wmz4B2nHLLgMqUaQaD9GVypLKz9gB1fLycsonRVi+SiQQJgTFYd/6H/oD4q0iYPBpsgtWRSXMfZmd50BR0UP/veEGcxf3d7XHn5K+T9B1+Kk7m3b7YMvNPG5KQApTcuov9WfuamntF7HS6LE0PuOykUE5sF8KAhC9gpOmEFs3QSYmce9PW/NNgHBdLiQ6iilfudKRWnA5c5U0eDm4v4DuRaknjt3WckqsCWREvMZL78wA1R6unUW5An3pU/JIFM2nSDYbLtbX0dqB4GZjkLZg7LGKLZgoeLm2IbRXiJ8p9XshXf+3Yn7b1NINCRB4HSnnj175UI0iXDACtvmSeNNHgJLFYUQME0G3iPwyfAaNbU4fC3xYWmmmrg47AAX7e66vWulsqMO5su6j0mNL+229owIWBYptpd1KF7qbiy/ysE= 2022-08-15T19:06:56.130 INFO:teuthology.kill:2022-08-15 19:06:56,127.127 INFO:teuthology.nuke:Checking targets against current locks 2022-08-15T19:06:56.192 INFO:teuthology.kill:2022-08-15 19:06:56,192.192 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-15T19:06:56.206 INFO:teuthology.kill:2022-08-15 19:06:56,205.205 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-15T19:06:56.270 INFO:teuthology.kill:2022-08-15 19:06:56,270.270 INFO:teuthology.orchestra.console:Power off smithi083 2022-08-15T19:06:56.319 INFO:teuthology.kill:2022-08-15 19:06:56,319.319 INFO:teuthology.orchestra.console:Power off smithi109 2022-08-15T19:07:04.660 INFO:teuthology.kill:2022-08-15 19:07:04,660.660 INFO:teuthology.orchestra.console:Power off for smithi083 completed 2022-08-15T19:07:04.685 INFO:teuthology.kill:2022-08-15 19:07:04,685.685 INFO:teuthology.orchestra.console:Power off for smithi109 completed 2022-08-15T19:07:04.826 INFO:teuthology.kill:2022-08-15 19:07:04,825.825 INFO:teuthology.lock.ops:unlocked: smithi083.front.sepia.ceph.com 2022-08-15T19:07:04.842 INFO:teuthology.kill:2022-08-15 19:07:04,842.842 INFO:teuthology.lock.ops:unlocked: smithi109.front.sepia.ceph.com 2022-08-15T19:09:05.084 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-08-15T19:09:05.198 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi083 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-08-15_18:43:38-rados-wip-yuri4-testing-2022-08-15-0951-distro-default-smithi/6974011 2022-08-15T19:09:05.198 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi109 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-08-15_18:43:38-rados-wip-yuri4-testing-2022-08-15-0951-distro-default-smithi/6974011