2021-10-21T14:22:28.986 INFO:root:teuthology version: 1.1.0-c56135d1 2021-10-21T14:22:29.118 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi023.front.sepia.ceph.com' reimaging 2021-10-21T14:22:29.119 INFO:teuthology.lock.ops:Updating [ubuntu@smithi023.front.sepia.ceph.com]: reset os type and version on server 2021-10-21T14:22:29.123 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2021-10-21T14:22:29.145 INFO:teuthology.lock.ops:Node 'ubuntu@smithi023.front.sepia.ceph.com' reimaging is complete 2021-10-21T14:22:29.146 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi039.front.sepia.ceph.com' reimaging 2021-10-21T14:22:29.146 INFO:teuthology.lock.ops:Updating [ubuntu@smithi039.front.sepia.ceph.com]: reset os type and version on server 2021-10-21T14:22:29.147 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2021-10-21T14:22:29.165 INFO:teuthology.lock.ops:Node 'ubuntu@smithi039.front.sepia.ceph.com' reimaging is complete 2021-10-21T14:22:29.376 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of centos 8.2 2021-10-21T14:22:29.385 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 8.2 2021-10-21T14:22:29.945 INFO:teuthology.orchestra.console:Power off smithi023 2021-10-21T14:22:29.966 INFO:teuthology.orchestra.console:Power off smithi039 2021-10-21T14:22:38.339 INFO:teuthology.orchestra.console:Power off for smithi039 completed 2021-10-21T14:22:38.441 INFO:teuthology.orchestra.console:Power on smithi039 2021-10-21T14:22:42.449 INFO:teuthology.orchestra.console:Power off for smithi023 completed 2021-10-21T14:22:42.552 INFO:teuthology.orchestra.console:Power on smithi023 2021-10-21T14:22:46.817 INFO:teuthology.orchestra.console:Power on for smithi039 completed 2021-10-21T14:22:46.918 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish 2021-10-21T14:22:50.924 INFO:teuthology.orchestra.console:Power on for smithi023 completed 2021-10-21T14:22:51.026 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish 2021-10-21T14:25:33.957 ERROR:teuthology.orchestra.connection:Error authenticating with smithi039.front.sepia.ceph.com: Authentication failed. 2021-10-21T14:27:50.316 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-10-21T14:27:50.618 INFO:teuthology.provision.fog.smithi039:Node is ready 2021-10-21T14:27:50.673 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039.front.sepia.ceph.com 2021-10-21T14:27:50.729 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39 smithi039.front.sepia.ceph.com smithi039 2021-10-21T14:27:51.056 INFO:teuthology.provision.fog.smithi039:Deploy complete! 2021-10-21T14:28:11.321 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-10-21T14:28:11.618 INFO:teuthology.provision.fog.smithi023:Node is ready 2021-10-21T14:28:11.676 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com 2021-10-21T14:28:11.733 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023 2021-10-21T14:28:12.062 INFO:teuthology.provision.fog.smithi023:Deploy complete! 2021-10-21T14:28:12.222 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com 2021-10-21T14:28:12.223 INFO:teuthology.lock.ops:New key found. Updating... 2021-10-21T14:28:12.245 INFO:teuthology.lock.ops:Checking smithi039.front.sepia.ceph.com 2021-10-21T14:28:12.245 INFO:teuthology.lock.ops:New key found. Updating... 2021-10-21T14:28:12.264 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server 2021-10-21T14:28:12.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2021-10-21T14:28:12.523 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2021-10-21T14:28:12.578 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Linux" 2021-10-21T14:28:12.578 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="8 (Core)" 2021-10-21T14:28:12.578 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2021-10-21T14:28:12.579 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2021-10-21T14:28:12.579 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="8" 2021-10-21T14:28:12.579 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el8" 2021-10-21T14:28:12.579 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-10-21T14:28:12.580 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2021-10-21T14:28:12.580 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-10-21T14:28:12.580 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://www.centos.org/" 2021-10-21T14:28:12.580 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-10-21T14:28:12.581 INFO:teuthology.orchestra.run.smithi023.stdout: 2021-10-21T14:28:12.581 INFO:teuthology.orchestra.run.smithi023.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-10-21T14:28:12.581 INFO:teuthology.orchestra.run.smithi023.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-10-21T14:28:12.582 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-10-21T14:28:12.582 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-10-21T14:28:12.582 INFO:teuthology.orchestra.run.smithi023.stdout: 2021-10-21T14:28:12.584 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2021-10-21T14:28:12.602 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: set os type and version on server 2021-10-21T14:28:12.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2021-10-21T14:28:12.877 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2021-10-21T14:28:12.935 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Linux" 2021-10-21T14:28:12.935 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8 (Core)" 2021-10-21T14:28:12.935 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2021-10-21T14:28:12.936 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2021-10-21T14:28:12.936 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8" 2021-10-21T14:28:12.936 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2021-10-21T14:28:12.936 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-10-21T14:28:12.937 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2021-10-21T14:28:12.937 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-10-21T14:28:12.937 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.centos.org/" 2021-10-21T14:28:12.937 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-10-21T14:28:12.937 INFO:teuthology.orchestra.run.smithi039.stdout: 2021-10-21T14:28:12.938 INFO:teuthology.orchestra.run.smithi039.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-10-21T14:28:12.938 INFO:teuthology.orchestra.run.smithi039.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-10-21T14:28:12.938 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-10-21T14:28:12.938 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-10-21T14:28:12.939 INFO:teuthology.orchestra.run.smithi039.stdout: 2021-10-21T14:28:12.939 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2021-10-21T14:28:13.028 INFO:teuthology.dispatcher.supervisor:Running job 6454885 2021-10-21T14:28:13.029 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-10-21_13:40:38-rados-wip-yuri2-testing-2021-10-20-1700-pacific-distro-basic-smithi/6454885 --name yuriw-2021-10-21_13:40:38-rados-wip-yuri2-testing-2021-10-20-1700-pacific-distro-basic-smithi --description rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} -- /home/teuthworker/archive/yuriw-2021-10-21_13:40:38-rados-wip-yuri2-testing-2021-10-20-1700-pacific-distro-basic-smithi/6454885/orig.config.yaml 2021-10-21T14:28:13.032 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-10-21_13:40:38-rados-wip-yuri2-testing-2021-10-20-1700-pacific-distro-basic-smithi/6454885 2021-10-21T14:28:13.033 INFO:teuthology.dispatcher.supervisor:Job PID: 4528 2021-10-21T14:28:13.033 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-10-22T02:28:37.165 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-10-22T02:28:37.463 INFO:teuthology.kill:Killing Pids: {4528} 2021-10-22T02:28:38.720 INFO:teuthology.task.internal:roles: ubuntu@smithi023.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-10-22T02:28:38.721 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-10-22T02:28:38.722 INFO:teuthology.misc:Compressing logs... 2021-10-22T02:28:38.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2021-10-22T02:28:38.932 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2021-10-22T02:28:39.312 INFO:teuthology.misc:Compressing logs... 2021-10-22T02:28:40.344 INFO:teuthology.orchestra.run.smithi023.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-10-22T02:28:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-10-22T02:28:41.306 INFO:teuthology.misc:Compressing logs... 2021-10-22T02:28:54.783 INFO:teuthology.kill:No teuthology processes running 2021-10-22T02:28:55.980 INFO:teuthology.kill:Nuking machines: ['smithi023', 'smithi039'] 2021-10-22T02:28:56.999 INFO:teuthology.kill:2021-10-22 02:28:56,999.999 INFO:teuthology.nuke:targets: 2021-10-22T02:28:57.000 INFO:teuthology.kill: smithi023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0lv262yBblkTHv5H/4Rxwf7qqJnAByO6Cods1QQvwzYGlsj0BhDcEzTxwMMzTxwPzPI6lXo9zckQmtGlsL7bdJg9pYubGd83rwAEUAqdkV4Hsryo1mwr8GwL53RUuN5COZU3Z3Z1R8kax9X2YtyUfw8YNBDsed2/4zj9TaPqY7iyO/EPvCViszSeGOLIwanIqQ36kj56icU37TAekJ8MNHkwudY2v+ImisgkgGok8VF3im+f60JcYGC8x+2AzkRvy59At4G9Tw3wzwHpxV4KWcFVF6U7cz1z0l8/hVlbsXyWB/AfyuebTolYNJ/w/FQJZSD8+ULBvsnFjJ+v4p9wVS5PMMid8vD3PCbVPBuCmgEM3KtcY0v7u/iN4j3Qi/Wuckyl/n/z2WVpyjcZYNJ8aYCNfIKntfVCMfnM0KG1aSHQ26MDFK7BhcrrOaDkj1jEdYKQDhBKdWuJ2Aj+8wT/xVt/4RS841jsH9q72Npp9WJgsRpwSGIXONlsxMlviVvk= 2021-10-22T02:28:57.001 INFO:teuthology.kill: smithi039.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC87GL92pfGkzYpIMk2d7o4zlya9HGLP/mEAGLSo2hWT5nIsP5oW5tUxaPCn4qT0sCyKI15oUSyTqRq3pgIhQD/x/bBfCtf9DqIEjsmkYT28+eZ1D9xxJ+RjGgGs1D/HYox88/GWtPdXdZCoRPAi0S++QAVAsV6v2qxint14q0oW2r626djsk+yZz8HFOa68mNUGHf4Xzdp9jW7AwSL9t3mu16sOJOfV6LHay2sViJjMk0jkk5H0suXm3+N/DG4JCvLY7EcXOrcUZQKmUMvLNMFXWq6QUuWBIdpvkQBR23VT0h+ecI1pRaAC99BByrso13HGFnJWot7ULYaYswKMlG/MmKNtDDQFk0eKndJk/IhYEqrkTBlANyYQ7UFelDi0pAa0m0c+9Nz54UcSfi1j68uhsMOa5hBlJreFb8ZlwCC0xeG+EXgElj0chmof+ItBexPMfyHEfacQEa0de4yFBt++7M49nnEoZl8kErXCYt8rOURnkpXKSIF8Ncok5AXT8s= 2021-10-22T02:28:57.029 INFO:teuthology.kill:2021-10-22 02:28:57,029.029 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-10-22T02:28:57.034 INFO:teuthology.kill:2021-10-22 02:28:57,034.034 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-10-22T02:28:57.110 INFO:teuthology.kill:2021-10-22 02:28:57,109.109 INFO:teuthology.orchestra.console:Power off smithi039 2021-10-22T02:28:57.124 INFO:teuthology.kill:2021-10-22 02:28:57,124.124 INFO:teuthology.orchestra.console:Power off smithi023 2021-10-22T02:29:05.478 INFO:teuthology.kill:2021-10-22 02:29:05,478.478 INFO:teuthology.orchestra.console:Power off for smithi039 completed 2021-10-22T02:29:05.489 INFO:teuthology.kill:2021-10-22 02:29:05,489.489 INFO:teuthology.orchestra.console:Power off for smithi023 completed 2021-10-22T02:29:05.637 INFO:teuthology.kill:2021-10-22 02:29:05,636.636 INFO:teuthology.lock.ops:unlocked: smithi039.front.sepia.ceph.com 2021-10-22T02:29:05.640 INFO:teuthology.kill:2021-10-22 02:29:05,640.640 INFO:teuthology.lock.ops:unlocked: smithi023.front.sepia.ceph.com 2021-10-22T02:31:05.870 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-10-22T02:31:05.976 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi023 but it was locked by another job: /home/teuthworker/archive/trociny-2021-10-21_12:56:56-rbd-master-distro-basic-smithi/6454776 2021-10-22T02:31:05.977 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi039 but it was locked by another job: /home/teuthworker/archive/trociny-2021-10-21_12:56:56-rbd-master-distro-basic-smithi/6454776