2022-07-01T20:02:31.788 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-07-01T20:02:31.983 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2022-07-01T20:02:31.984 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2022-07-01T20:02:31.985 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2022-07-01T20:02:32.021 INFO:teuthology.lock.ops:Node 'smithi112.front.sepia.ceph.com' reimaging is complete 2022-07-01T20:02:32.022 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging 2022-07-01T20:02:32.023 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server 2022-07-01T20:02:32.023 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2022-07-01T20:02:32.079 INFO:teuthology.lock.ops:Node 'smithi167.front.sepia.ceph.com' reimaging is complete 2022-07-01T20:02:32.277 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 8.stream 2022-07-01T20:02:32.319 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 8.stream 2022-07-01T20:02:32.597 INFO:teuthology.orchestra.console:Power off smithi112 2022-07-01T20:02:32.648 INFO:teuthology.orchestra.console:Power off smithi167 2022-07-01T20:02:41.026 INFO:teuthology.orchestra.console:Power off for smithi167 completed 2022-07-01T20:02:41.128 INFO:teuthology.orchestra.console:Power on smithi167 2022-07-01T20:02:45.127 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2022-07-01T20:02:45.229 INFO:teuthology.orchestra.console:Power on smithi112 2022-07-01T20:02:49.558 INFO:teuthology.orchestra.console:Power on for smithi167 completed 2022-07-01T20:02:49.661 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish 2022-07-01T20:02:53.655 INFO:teuthology.orchestra.console:Power on for smithi112 completed 2022-07-01T20:02:53.758 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2022-07-01T20:07:06.645 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-01T20:07:32.228 INFO:teuthology.provision.fog.smithi112:Node is ready 2022-07-01T20:07:32.253 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2022-07-01T20:07:32.315 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2022-07-01T20:07:32.706 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2022-07-01T20:07:42.867 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-01T20:07:43.184 INFO:teuthology.provision.fog.smithi167:Node is ready 2022-07-01T20:07:43.202 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com 2022-07-01T20:07:43.262 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167 2022-07-01T20:07:43.634 INFO:teuthology.provision.fog.smithi167:Deploy complete! 2022-07-01T20:07:43.814 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2022-07-01T20:07:43.815 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-01T20:07:43.840 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com 2022-07-01T20:07:43.841 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-01T20:07:43.861 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2022-07-01T20:07:43.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2022-07-01T20:07:44.127 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2022-07-01T20:07:44.184 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2022-07-01T20:07:44.184 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8" 2022-07-01T20:07:44.185 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2022-07-01T20:07:44.185 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2022-07-01T20:07:44.185 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8" 2022-07-01T20:07:44.185 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2022-07-01T20:07:44.186 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-01T20:07:44.186 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2022-07-01T20:07:44.186 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-01T20:07:44.186 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2022-07-01T20:07:44.187 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-01T20:07:44.187 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-01T20:07:44.187 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-01T20:07:44.188 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2022-07-01T20:07:44.210 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server 2022-07-01T20:07:44.211 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2022-07-01T20:07:44.412 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2022-07-01T20:07:44.470 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2022-07-01T20:07:44.470 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8" 2022-07-01T20:07:44.470 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2022-07-01T20:07:44.471 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2022-07-01T20:07:44.471 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8" 2022-07-01T20:07:44.471 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2022-07-01T20:07:44.471 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-01T20:07:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2022-07-01T20:07:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-01T20:07:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2022-07-01T20:07:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-01T20:07:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-01T20:07:44.473 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-01T20:07:44.473 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2022-07-01T20:07:44.590 INFO:teuthology.dispatcher.supervisor:Running job 6909584 2022-07-01T20:07:44.591 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/bin/teuthology -v --owner scheduled_ksirivad@teuthology --archive /home/teuthworker/archive/ksirivad-2022-07-01_18:57:52-rados:thrash-erasure-code-main-distro-default-smithi/6909584 --name ksirivad-2022-07-01_18:57:52-rados:thrash-erasure-code-main-distro-default-smithi --description rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/ksirivad-2022-07-01_18:57:52-rados:thrash-erasure-code-main-distro-default-smithi/6909584/orig.config.yaml 2022-07-01T20:07:44.596 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ksirivad-2022-07-01_18:57:52-rados:thrash-erasure-code-main-distro-default-smithi/6909584 2022-07-01T20:07:44.597 INFO:teuthology.dispatcher.supervisor:Job PID: 21045 2022-07-01T20:07:44.598 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-02T08:08:43.685 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-02T08:08:43.882 INFO:teuthology.kill:Killing Pids: {21045} 2022-07-02T08:08:45.368 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-07-02T08:08:45.369 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-07-02T08:08:45.370 INFO:teuthology.misc:Compressing logs... 2022-07-02T08:08:45.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2022-07-02T08:08:45.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2022-07-02T08:08:45.927 INFO:teuthology.misc:Compressing logs... 2022-07-02T08:08:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-07-02T08:08:47.063 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-07-02T08:08:47.722 INFO:teuthology.misc:Compressing logs... 2022-07-02T08:08:54.174 INFO:teuthology.kill:No teuthology processes running 2022-07-02T08:08:55.654 INFO:teuthology.kill:Nuking machines: ['smithi112', 'smithi167'] 2022-07-02T08:08:56.783 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-02T08:08:56.784 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-02T08:08:56.785 INFO:teuthology.kill:2022-07-02 08:08:56,783.783 INFO:teuthology.nuke:targets: 2022-07-02T08:08:56.785 INFO:teuthology.kill: smithi112.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwI95ItlTHHlY07F5ZgPzhij2st6haD1zPzp7wZR7jTeN4Dm7jgqqgPAG3y0uvhdABL+Bai2a/4fqlOH50QmAOertvCsrA5/QulQNpt5ruy9i9du3eHqHBy/BjjKJHeo8D8whXLp3m1x5SOsakMBaBtjmyDHkpSmzo2ZFrQkrAiMqHChMeuvpM1JzcnxNf/WWNUL4WwK+j7ZVBHE5CTtMKNo2LMjb3/N2gu19GFvCzaOroNy5kHCpIwnKWJcJJ6dMdqrkyTpBT1WjFQ3S2BJYib/O2JYHp9gOsmdqfbcGumFZ+/3YhPE5L1+hWYK11zITHFHBh7Xt95JJz/SsjQlve6JvK9GcksaiFX53D4LlTwzYFuBoROym2IPLmGprKC34RjK1Qgvtj7Eo+bgS+3jZjpuhd/01qSvcyJE8z138cha5eZ7lOB4Xu21JO4cDY5mcNr9sCt/cG/b1nHIBKO+WGUa/5b4dU5q2ypu0WPWz1HIwBHiEmmih9BnNVcZBaP60= 2022-07-02T08:08:56.785 INFO:teuthology.kill: smithi167.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2WFvRCdzQkv1So2Bdp6JovpVB3ktWxE6wDkXB3VvSYggeE9JV8mMYD7UNkTuL/BUhhpydTAAb/z4qA4m4fRviDEi0AXYAaaTCMn2KuhcWpBFrvcXZwXMbKBUnGmEMglntzuxAoRLNhwoyctph6mdgeu/QvndWkFZgOKQuqpSOL1gWsvaHkJhpdI0J25E/NeuiHZeCR5ToZOZ2EU+FDZsiDRvFUDFR8ggnv5fD5jT6M58nR3SX5ztOxVZS03RdeLUfvflyvt6UZ0j1hOLgl3SCTNYIg/KX+9OMQ49eoUExoT7soK/FuTfnUnM2KEi3iZFWVTnbsXq8LJMg4XkaAgLPF4u7i2kynmUaBttbVtGcQ6AoOZlYDL00aqOfT3a+KIhuh+iQ4ep15y3XzOp86zEkNbJJXjLbIhAmrlqWp7C55C91zNzmRNX6M4NncDI0+OtIVP33Z6FzvRLyq3YfDCnQWimZBXYjQjqphsGenijXYs61BhykOeaez49lA3IE6eM= 2022-07-02T08:08:56.785 INFO:teuthology.kill:2022-07-02 08:08:56,783.783 INFO:teuthology.nuke:Checking targets against current locks 2022-07-02T08:08:56.848 INFO:teuthology.kill:2022-07-02 08:08:56,848.848 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-02T08:08:56.879 INFO:teuthology.kill:2022-07-02 08:08:56,878.878 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-02T08:08:56.945 INFO:teuthology.kill:2022-07-02 08:08:56,945.945 INFO:teuthology.orchestra.console:Power off smithi112 2022-07-02T08:08:56.981 INFO:teuthology.kill:2022-07-02 08:08:56,981.981 INFO:teuthology.orchestra.console:Power off smithi167 2022-07-02T08:09:05.316 INFO:teuthology.kill:2022-07-02 08:09:05,315.315 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2022-07-02T08:09:05.476 INFO:teuthology.kill:2022-07-02 08:09:05,475.475 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2022-07-02T08:09:09.451 INFO:teuthology.kill:2022-07-02 08:09:09,450.450 INFO:teuthology.orchestra.console:Power off for smithi167 completed 2022-07-02T08:09:09.611 INFO:teuthology.kill:2022-07-02 08:09:09,611.611 INFO:teuthology.lock.ops:unlocked: smithi167.front.sepia.ceph.com 2022-07-02T08:11:09.950 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-02T08:11:10.081 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi112 but it was locked by another job: /home/teuthworker/archive/nshelke-2022-07-02_04:41:48-fs:volumes-wip-nshelke-filter-internal-dirs-distro-default-smithi/6910291 2022-07-02T08:11:10.081 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi167 but it was locked by another job: /home/teuthworker/archive/nshelke-2022-07-02_04:41:48-fs:volumes-wip-nshelke-filter-internal-dirs-distro-default-smithi/6910291