2022-06-15T01:07:03.123 INFO:root:teuthology version: 1.1.1.dev370+g2290146e 2022-06-15T01:07:03.272 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging 2022-06-15T01:07:03.274 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server 2022-06-15T01:07:03.274 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-06-15T01:07:03.303 INFO:teuthology.lock.ops:Node 'smithi002.front.sepia.ceph.com' reimaging is complete 2022-06-15T01:07:03.304 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging 2022-06-15T01:07:03.305 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server 2022-06-15T01:07:03.306 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2022-06-15T01:07:03.332 INFO:teuthology.lock.ops:Node 'smithi104.front.sepia.ceph.com' reimaging is complete 2022-06-15T01:07:03.552 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of centos 8.stream 2022-06-15T01:07:03.562 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 8.stream 2022-06-15T01:07:04.063 INFO:teuthology.orchestra.console:Power off smithi104 2022-06-15T01:07:04.084 INFO:teuthology.orchestra.console:Power off smithi002 2022-06-15T01:07:12.467 INFO:teuthology.orchestra.console:Power off for smithi104 completed 2022-06-15T01:07:12.478 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2022-06-15T01:07:12.569 INFO:teuthology.orchestra.console:Power on smithi104 2022-06-15T01:07:12.581 INFO:teuthology.orchestra.console:Power on smithi002 2022-06-15T01:07:16.834 INFO:teuthology.orchestra.console:Power on for smithi104 completed 2022-06-15T01:07:16.935 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish 2022-06-15T01:07:20.999 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2022-06-15T01:07:21.100 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2022-06-15T01:10:54.523 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-15T01:11:14.667 INFO:teuthology.provision.fog.smithi002:Node is ready 2022-06-15T01:11:14.700 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2022-06-15T01:11:14.762 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2022-06-15T01:11:15.180 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2022-06-15T01:11:49.509 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-15T01:11:49.789 INFO:teuthology.provision.fog.smithi104:Node is ready 2022-06-15T01:11:49.807 INFO:teuthology.orchestra.run.smithi104.stdout:smithi104.front.sepia.ceph.com 2022-06-15T01:11:49.869 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.104 smithi104.front.sepia.ceph.com smithi104 2022-06-15T01:11:50.221 INFO:teuthology.provision.fog.smithi104:Deploy complete! 2022-06-15T01:11:50.444 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2022-06-15T01:11:50.446 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-15T01:11:50.472 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com 2022-06-15T01:11:50.472 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-15T01:11:50.504 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2022-06-15T01:11:50.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-06-15T01:11:50.748 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2022-06-15T01:11:50.807 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2022-06-15T01:11:50.808 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8" 2022-06-15T01:11:50.808 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2022-06-15T01:11:50.809 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2022-06-15T01:11:50.809 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8" 2022-06-15T01:11:50.809 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2022-06-15T01:11:50.810 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-15T01:11:50.810 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2022-06-15T01:11:50.811 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-15T01:11:50.811 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2022-06-15T01:11:50.812 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-15T01:11:50.812 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-15T01:11:50.813 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-15T01:11:50.815 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-06-15T01:11:50.838 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server 2022-06-15T01:11:50.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2022-06-15T01:11:51.072 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2022-06-15T01:11:51.131 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2022-06-15T01:11:51.131 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="8" 2022-06-15T01:11:51.131 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2022-06-15T01:11:51.131 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2022-06-15T01:11:51.132 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="8" 2022-06-15T01:11:51.132 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el8" 2022-06-15T01:11:51.133 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-15T01:11:51.133 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2022-06-15T01:11:51.133 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-15T01:11:51.133 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2022-06-15T01:11:51.133 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-15T01:11:51.134 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-15T01:11:51.134 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-15T01:11:51.135 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2022-06-15T01:11:51.274 INFO:teuthology.dispatcher.supervisor:Running job 6878870 2022-06-15T01:11:51.275 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_2290146eac7577b8500f128a53856c3ea4a00e3c/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-06-14_21:33:15-rados:thrash-erasure-code-main-distro-default-smithi/6878870 --name lflores-2022-06-14_21:33:15-rados:thrash-erasure-code-main-distro-default-smithi --description rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/lflores-2022-06-14_21:33:15-rados:thrash-erasure-code-main-distro-default-smithi/6878870/orig.config.yaml 2022-06-15T01:11:51.280 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-06-14_21:33:15-rados:thrash-erasure-code-main-distro-default-smithi/6878870 2022-06-15T01:11:51.281 INFO:teuthology.dispatcher.supervisor:Job PID: 3442 2022-06-15T01:11:51.282 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-06-15T13:13:00.131 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-06-15T13:13:00.533 INFO:teuthology.kill:Killing Pids: {3442} 2022-06-15T13:13:05.447 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-06-15T13:13:05.448 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-06-15T13:13:05.449 INFO:teuthology.misc:Compressing logs... 2022-06-15T13:13:05.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-06-15T13:13:05.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2022-06-15T13:13:06.238 INFO:teuthology.misc:Compressing logs... 2022-06-15T13:13:07.260 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-15T13:13:07.507 INFO:teuthology.orchestra.run.smithi104.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-15T13:13:08.558 INFO:teuthology.misc:Compressing logs... 2022-06-15T13:13:36.030 INFO:teuthology.kill:No teuthology processes running 2022-06-15T13:13:40.569 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi104'] 2022-06-15T13:13:44.445 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_2290146eac7577b8500f128a53856c3ea4a00e3c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-06-15T13:13:44.446 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-06-15T13:13:44.447 INFO:teuthology.kill:2022-06-15 13:13:44,444.444 INFO:teuthology.nuke:targets: 2022-06-15T13:13:44.447 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPlNTb6GnMY5pmXacK+ZItJ+r7ilwIYxWEiPGiVmfVspMsgF8pH4tYxuSqZIJGKMc6kyVFknsLxF/BPxaF9C+M52m2iIsw1w4X/2yQW5RWHWq5zASEgUOlLQDJQGbYt2uyp3iaiBjUEOfyWCfH0pY9i7WtWJQCuc/FingRlAe8OPd46t5shbb0ocumyljH6Lpc7SSEW8fnfY1yY7RwS6uY0pQOkz8y18El3BU8dq7fk0uh1OWag2EeUttBpGC/bZK1IN9lpCXDEkZwQMyMr6RHWQh2laYk5AX5fBzzg393i8GjI4eC1KHwTV8EybsixXEE7zYUMxkBO9WounzQr1z8qj8j2m0rY1A93ao0HAEvbEagV+nk7SR1m/UGcM9IvzkV9CCV6pfyZERCgzAR1QpEdKzQMgAGJersl9aO9mJcch1Vy3eG3OMCLYDEE1ZLGq8XdGIPFIYSS+MwYoy0OsnZUkHC3Tq0CGDNCCer346pN89g4gN4CQ+TQtKBy5AMW8E= 2022-06-15T13:13:44.448 INFO:teuthology.kill: smithi104.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkx2kgl3fmd8PKcp90tDCnQhWS9UxgD3nNhaOuTeKBJXnHv5kn5xCwbA+PdxTaRmlRvHNznefugDDkwdevKqiEVk9GmTlCMlmbBTPl0dR8hhiTbr45l120GZE23P+wiLPhr4FJ3Nbj0sBkr2Eu6yxLry2yx2sn+Oj/NPaxbdZEJnZ+Z6OWGukGMiNz1lFv8SMD01XF9lXvfgjvjHQDjWVmnrzXzo/aRusXqdSIs49RdMwwq84TeSQ5vCnYR0Cqh+ZzSbt8BnCgfRNoERkMVzewWexVAQQ08A3ZjuImI/trj580+F49P2zaG3ALgxIFhWPwL8MdNo1h5s6WFrUs8Teg3lGEz8eeEuWsvq7H85QhtnOnhZ9IN6Bc2ObGQF9Wo3qNyRlk9OdyxxwDLcbZ7I/18xLVRFjZr+4LeL8vF/EQ2KG33KtBmE3zGFoTB3GuETKp3AsNEfXYm2jedMo7vWSyJi7i2QHJbJeZ0V102Eo3rSF/snmrzRs3p1cy8bgbKs8= 2022-06-15T13:13:44.476 INFO:teuthology.kill:2022-06-15 13:13:44,476.476 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-15T13:13:44.480 INFO:teuthology.kill:2022-06-15 13:13:44,480.480 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-15T13:13:44.652 INFO:teuthology.kill:2022-06-15 13:13:44,647.647 INFO:teuthology.orchestra.console:Power off smithi104 2022-06-15T13:13:44.756 INFO:teuthology.kill:2022-06-15 13:13:44,755.755 INFO:teuthology.orchestra.console:Power off smithi002 2022-06-15T13:13:53.264 INFO:teuthology.kill:2022-06-15 13:13:53,264.264 INFO:teuthology.orchestra.console:Power off for smithi104 completed 2022-06-15T13:13:53.272 INFO:teuthology.kill:2022-06-15 13:13:53,271.271 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2022-06-15T13:13:53.439 INFO:teuthology.kill:2022-06-15 13:13:53,438.438 INFO:teuthology.lock.ops:unlocked: smithi104.front.sepia.ceph.com 2022-06-15T13:13:53.448 INFO:teuthology.kill:2022-06-15 13:13:53,446.446 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com 2022-06-15T13:15:54.011 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-06-15T13:15:54.249 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi002 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-06-15_03:30:04-rados-octopus-distro-default-smithi/6879302 2022-06-15T13:15:54.254 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi104 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-06-15_03:30:04-rados-octopus-distro-default-smithi/6879303