2021-09-27T16:56:37.957 INFO:root:teuthology version: 1.1.0-f08b273e 2021-09-27T16:56:38.123 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi050.front.sepia.ceph.com' reimaging 2021-09-27T16:56:38.124 INFO:teuthology.lock.ops:Updating [ubuntu@smithi050.front.sepia.ceph.com]: reset os type and version on server 2021-09-27T16:56:38.125 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2021-09-27T16:56:38.144 INFO:teuthology.lock.ops:Node 'ubuntu@smithi050.front.sepia.ceph.com' reimaging is complete 2021-09-27T16:56:38.144 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi103.front.sepia.ceph.com' reimaging 2021-09-27T16:56:38.145 INFO:teuthology.lock.ops:Updating [ubuntu@smithi103.front.sepia.ceph.com]: reset os type and version on server 2021-09-27T16:56:38.145 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2021-09-27T16:56:38.163 INFO:teuthology.lock.ops:Node 'ubuntu@smithi103.front.sepia.ceph.com' reimaging is complete 2021-09-27T16:56:38.164 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging 2021-09-27T16:56:38.164 INFO:teuthology.lock.ops:Updating [ubuntu@smithi153.front.sepia.ceph.com]: reset os type and version on server 2021-09-27T16:56:38.164 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2021-09-27T16:56:38.183 INFO:teuthology.lock.ops:Node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging is complete 2021-09-27T16:56:38.378 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 8.3 2021-09-27T16:56:38.391 INFO:teuthology.provision.fog.smithi103:Scheduling deploy of centos 8.3 2021-09-27T16:56:38.426 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 8.3 2021-09-27T16:56:38.744 INFO:teuthology.orchestra.console:Power off smithi050 2021-09-27T16:56:38.764 INFO:teuthology.orchestra.console:Power off smithi103 2021-09-27T16:56:38.834 INFO:teuthology.orchestra.console:Power off smithi153 2021-09-27T16:56:47.119 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2021-09-27T16:56:47.126 INFO:teuthology.orchestra.console:Power off for smithi103 completed 2021-09-27T16:56:47.221 INFO:teuthology.orchestra.console:Power on smithi050 2021-09-27T16:56:47.228 INFO:teuthology.orchestra.console:Power on smithi103 2021-09-27T16:56:51.322 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-09-27T16:56:51.423 INFO:teuthology.orchestra.console:Power on smithi153 2021-09-27T16:56:55.601 INFO:teuthology.orchestra.console:Power on for smithi103 completed 2021-09-27T16:56:55.677 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2021-09-27T16:56:55.704 INFO:teuthology.provision.fog.smithi103:Waiting for deploy to finish 2021-09-27T16:56:55.725 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2021-09-27T16:56:55.778 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2021-09-27T16:56:55.826 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2021-09-27T17:01:03.514 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-27T17:01:03.829 INFO:teuthology.provision.fog.smithi050:Node is ready 2021-09-27T17:01:03.845 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2021-09-27T17:01:03.901 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2021-09-27T17:01:04.230 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2021-09-27T17:02:00.963 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-27T17:02:01.017 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-27T17:02:01.265 INFO:teuthology.provision.fog.smithi103:Node is ready 2021-09-27T17:02:01.280 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103.front.sepia.ceph.com 2021-09-27T17:02:01.334 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103 smithi103.front.sepia.ceph.com smithi103 2021-09-27T17:02:01.341 INFO:teuthology.provision.fog.smithi153:Node is ready 2021-09-27T17:02:01.357 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2021-09-27T17:02:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2021-09-27T17:02:01.670 INFO:teuthology.provision.fog.smithi103:Deploy complete! 2021-09-27T17:02:01.734 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2021-09-27T17:02:01.957 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2021-09-27T17:02:01.958 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-27T17:02:01.977 INFO:teuthology.lock.ops:Checking smithi103.front.sepia.ceph.com 2021-09-27T17:02:01.977 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-27T17:02:01.994 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2021-09-27T17:02:01.995 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-27T17:02:02.017 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2021-09-27T17:02:02.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2021-09-27T17:02:02.291 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2021-09-27T17:02:02.346 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Linux" 2021-09-27T17:02:02.346 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8" 2021-09-27T17:02:02.347 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2021-09-27T17:02:02.347 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2021-09-27T17:02:02.347 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8" 2021-09-27T17:02:02.348 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8" 2021-09-27T17:02:02.348 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Linux 8" 2021-09-27T17:02:02.349 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2021-09-27T17:02:02.349 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-09-27T17:02:02.349 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2021-09-27T17:02:02.350 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-09-27T17:02:02.350 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-09-27T17:02:02.351 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-09-27T17:02:02.353 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2021-09-27T17:02:02.373 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: set os type and version on server 2021-09-27T17:02:02.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2021-09-27T17:02:02.636 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2021-09-27T17:02:02.691 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Linux" 2021-09-27T17:02:02.691 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="8" 2021-09-27T17:02:02.691 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2021-09-27T17:02:02.691 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2021-09-27T17:02:02.692 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="8" 2021-09-27T17:02:02.692 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el8" 2021-09-27T17:02:02.692 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Linux 8" 2021-09-27T17:02:02.693 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2021-09-27T17:02:02.693 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-09-27T17:02:02.693 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2021-09-27T17:02:02.694 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-09-27T17:02:02.694 INFO:teuthology.orchestra.run.smithi103.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-09-27T17:02:02.694 INFO:teuthology.orchestra.run.smithi103.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-09-27T17:02:02.695 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2021-09-27T17:02:02.714 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2021-09-27T17:02:02.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2021-09-27T17:02:02.984 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2021-09-27T17:02:03.038 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Linux" 2021-09-27T17:02:03.039 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8" 2021-09-27T17:02:03.039 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2021-09-27T17:02:03.039 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2021-09-27T17:02:03.039 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8" 2021-09-27T17:02:03.040 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2021-09-27T17:02:03.040 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Linux 8" 2021-09-27T17:02:03.040 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2021-09-27T17:02:03.040 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-09-27T17:02:03.040 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2021-09-27T17:02:03.041 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-09-27T17:02:03.041 INFO:teuthology.orchestra.run.smithi153.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-09-27T17:02:03.041 INFO:teuthology.orchestra.run.smithi153.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-09-27T17:02:03.042 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2021-09-27T17:02:03.132 INFO:teuthology.dispatcher.supervisor:Running job 6409391 2021-09-27T17:02:03.133 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/virtualenv/bin/teuthology -v --owner scheduled_ideepika@teuthology --archive /home/teuthworker/archive/ideepika-2021-09-27_14:52:49-rados-wip-deepika-testing-2021-09-27-1601-distro-basic-smithi/6409391 --name ideepika-2021-09-27_14:52:49-rados-wip-deepika-testing-2021-09-27-1601-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/ideepika-2021-09-27_14:52:49-rados-wip-deepika-testing-2021-09-27-1601-distro-basic-smithi/6409391/orig.config.yaml 2021-09-27T17:02:03.136 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ideepika-2021-09-27_14:52:49-rados-wip-deepika-testing-2021-09-27-1601-distro-basic-smithi/6409391 2021-09-27T17:02:03.137 INFO:teuthology.dispatcher.supervisor:Job PID: 31288 2021-09-27T17:02:03.137 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-09-28T05:03:16.709 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-09-28T05:03:16.944 INFO:teuthology.kill:Killing Pids: {31288} 2021-09-28T05:03:18.220 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2021-09-28T05:03:18.221 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2021-09-28T05:03:18.221 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2021-09-28T05:03:18.222 INFO:teuthology.misc:Compressing logs... 2021-09-28T05:03:18.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2021-09-28T05:03:18.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2021-09-28T05:03:18.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2021-09-28T05:03:19.116 INFO:teuthology.misc:Compressing logs... 2021-09-28T05:03:19.725 INFO:teuthology.orchestra.run.smithi103.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-09-28T05:03:19.920 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-09-28T05:03:20.007 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-09-28T05:03:20.724 INFO:teuthology.misc:Compressing logs... 2021-09-28T05:06:05.092 INFO:teuthology.kill:No teuthology processes running 2021-09-28T05:06:06.366 INFO:teuthology.kill:Nuking machines: ['smithi050', 'smithi103', 'smithi153'] 2021-09-28T05:06:07.389 INFO:teuthology.kill:2021-09-28 05:06:07,389.389 INFO:teuthology.nuke:targets: 2021-09-28T05:06:07.390 INFO:teuthology.kill: smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNZ0wqm2pEe5rvVWWghwuDxyYEwOFhpg65R7ho/uSZ5cdG8xuMVsOBjjBd42Kz9KKJ5Hxcxmk1Oln5pwfsGtIyWZA7pypPKBKqER3KPlLYq/kymk3G0lJWgIThjVKyXqMhbjVaTVdnW6tF+QgMPwLihXhbgaK1gjr9TIKqOqxkQFXxnaJLT+PAysvQYntkdBHivW2K0fgw1wDTZMPLBlqlQzjTqMBpjhj99drgoa6nDR3b60ji1f9F0d09zBbjlhT/xC+1UitjWfrz5RVLTDrhtc2Ww21KOB2FmCJr65+n0yiFuXzqlY9WghU9kI3VV87NCSlRPVTBhzaXKzJUSREMmiRzFsd4sfv1mMT8BNrw+FSUiWtVlm3fMHffKmAj3JvFXlcm5KhyYrNZQ9AgIr+VUzlkW762rBqhk7PRfehX4KTnSRQ3CvqEnDoBUhNJkkgFRqD+NJEr7DmYTFR3gv6/B9nlQapgDW5RskTFYrbY2dNMuf8Q4MQvhNciMlk4qT0= 2021-09-28T05:06:07.390 INFO:teuthology.kill: smithi103.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCggITtziPEZ6L3DwuXVJPGv853TNIc+b2x9l6z8VLJ1vXUe3AhKqSfIIArVXLkY8Vgqi5XkBRRqKblwkU4XH1KpPNr8JHEHt3ayb1O5fZC/sQnNtBOcO/RIXEBg9Om7CQoifHa7a4ZOR0ddhPVR98XQ3qQCduP/C1h3PGJgyp6JB5ybEeB5Fnah/zo+VdbfmvFZcs/RZpQBMczqGCPy2oUXDO/kOrCgUCGHh8Ljd4ruKk4nm4Vtr6v6s8vEj/Wd4quSQZTBe1PE2tt2wV8EhLxI/1B/T16QdYpemMxvYoZOS31Fqu8lSn6z8kWBFKznY7DaTv1oBUsFVNZY/79/1JTtFPoeSQ5s5adRVvGT3t8f3XxRi5dcnU12mM1nb3N2KXYyD/peQ/9kpj4/g0oZPK3a8ZvEwh62O73xRObEAUrcu7MlXj7HeGPAPDvyyA9ai342gVNYW4f+85rfCjK9uIsmTw2twqNfX6fx0RaPMrpDlu2ioblNtb7fAq9yLNYJo0= 2021-09-28T05:06:07.391 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzdZ6fifLuQ+9d93rE8CSdbr2mYmHgjvreWQN8K4mXcpEbSFh+EnJwY/8JEDUa31fbgbe2jFGHAe9TlV7jjViZinIJD/oiLbOmKOpbj35KDIMOCBrhs0kBJcm5CdRHkCx71qacROt3+7eUqjq8E7EBr416+qPQO3Xz5kfpjhE6vb1wJ9DYoPEI8yKw8XIYyto9mvFW1MpmdiPo/MarsCoI6iuFlTF19hfH8F74jsJhHqLQMVeCYAIsmB9kbQtpcaGnh4XhwpRZpQEGVoLYPgiWEigQvN4o6Wr/tjJ6duYqfrb51v94nbCO7d3gnkumJmANTjBLqJBieRDL9jtkZ/6/Koqu52FByoOL7pj0VetKeFPgHdfzgllj/Km+E+n2TfAaR9HEI6zkj9YZsJHSxSYJtHyNQRy3ZchPrhFaV4DFr7zR2RFqbNKN1hO3PfDikeKfFpQL8elUhcTn+ZYZekvA7xj4QNvLuinDbtI9hEPh+x/n4KCLtoTfaDXPx8DYadE= 2021-09-28T05:06:07.421 INFO:teuthology.kill:2021-09-28 05:06:07,421.421 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-28T05:06:07.425 INFO:teuthology.kill:2021-09-28 05:06:07,425.425 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-28T05:06:07.426 INFO:teuthology.kill:2021-09-28 05:06:07,426.426 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-28T05:06:07.519 INFO:teuthology.kill:2021-09-28 05:06:07,518.518 INFO:teuthology.orchestra.console:Power off smithi103 2021-09-28T05:06:07.537 INFO:teuthology.kill:2021-09-28 05:06:07,536.536 INFO:teuthology.orchestra.console:Power off smithi050 2021-09-28T05:06:07.549 INFO:teuthology.kill:2021-09-28 05:06:07,549.549 INFO:teuthology.orchestra.console:Power off smithi153 2021-09-28T05:06:15.922 INFO:teuthology.kill:2021-09-28 05:06:15,922.922 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2021-09-28T05:06:15.939 INFO:teuthology.kill:2021-09-28 05:06:15,939.939 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-09-28T05:06:16.079 INFO:teuthology.kill:2021-09-28 05:06:16,078.078 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com 2021-09-28T05:06:16.091 INFO:teuthology.kill:2021-09-28 05:06:16,091.091 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com 2021-09-28T05:06:20.042 INFO:teuthology.kill:2021-09-28 05:06:20,042.042 INFO:teuthology.orchestra.console:Power off for smithi103 completed 2021-09-28T05:06:20.206 INFO:teuthology.kill:2021-09-28 05:06:20,205.205 INFO:teuthology.lock.ops:unlocked: smithi103.front.sepia.ceph.com 2021-09-28T05:08:20.551 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-09-28T05:08:20.686 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-09-28T05:08:20.687 INFO:teuthology.nuke:Checking targets against current locks 2021-09-28T05:08:20.791 INFO:teuthology.nuke:Not nuking smithi050.front.sepia.ceph.com because description doesn't match 2021-09-28T05:08:20.791 INFO:teuthology.nuke:Not nuking smithi153.front.sepia.ceph.com because description doesn't match 2021-09-28T05:08:20.792 INFO:teuthology.nuke:Not nuking smithi103.front.sepia.ceph.com because description doesn't match