2022-07-09T05:55:19.953 INFO:root:teuthology version: 1.1.1.dev381+g10062088
2022-07-09T05:55:20.370 INFO:teuthology.lock.ops:Start node 'smithi185.front.sepia.ceph.com' reimaging
2022-07-09T05:55:20.372 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: reset os type and version on server
2022-07-09T05:55:20.372 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server
2022-07-09T05:55:20.420 INFO:teuthology.lock.ops:Node 'smithi185.front.sepia.ceph.com' reimaging is complete
2022-07-09T05:55:20.421 INFO:teuthology.lock.ops:Start node 'smithi203.front.sepia.ceph.com' reimaging
2022-07-09T05:55:20.421 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: reset os type and version on server
2022-07-09T05:55:20.422 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2022-07-09T05:55:20.472 INFO:teuthology.lock.ops:Node 'smithi203.front.sepia.ceph.com' reimaging is complete
2022-07-09T05:55:20.722 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of rhel 8.6
2022-07-09T05:55:20.743 INFO:teuthology.provision.fog.smithi203:Scheduling deploy of rhel 8.6
2022-07-09T05:55:21.075 INFO:teuthology.orchestra.console:Power off smithi185
2022-07-09T05:55:21.225 INFO:teuthology.orchestra.console:Power off smithi203
2022-07-09T05:55:21.403 INFO:teuthology.orchestra.console:Power off for smithi185 completed
2022-07-09T05:55:21.479 INFO:teuthology.orchestra.console:Power off for smithi203 completed
2022-07-09T05:55:21.506 INFO:teuthology.orchestra.console:Power on smithi185
2022-07-09T05:55:21.642 INFO:teuthology.orchestra.console:Power on smithi203
2022-07-09T05:55:26.054 INFO:teuthology.orchestra.console:Power on for smithi203 completed
2022-07-09T05:55:26.156 INFO:teuthology.provision.fog.smithi203:Waiting for deploy to finish
2022-07-09T05:55:30.067 INFO:teuthology.orchestra.console:Power on for smithi185 completed
2022-07-09T05:55:30.180 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish
2022-07-09T05:59:51.901 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-07-09T06:00:17.270 INFO:teuthology.provision.fog.smithi185:Node is ready
2022-07-09T06:00:17.304 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com
2022-07-09T06:00:17.384 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185
2022-07-09T06:00:17.924 INFO:teuthology.provision.fog.smithi185:Deploy complete!
2022-07-09T06:00:22.932 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-07-09T06:00:23.173 INFO:teuthology.provision.fog.smithi203:Node is ready
2022-07-09T06:00:23.207 INFO:teuthology.orchestra.run.smithi203.stdout:smithi203.front.sepia.ceph.com
2022-07-09T06:00:23.291 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.15.203 smithi203.front.sepia.ceph.com smithi203
2022-07-09T06:00:23.744 INFO:teuthology.provision.fog.smithi203:Deploy complete!
2022-07-09T06:00:24.112 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com
2022-07-09T06:00:24.113 INFO:teuthology.lock.ops:New key found. Updating...
2022-07-09T06:00:24.155 INFO:teuthology.lock.ops:Checking smithi203.front.sepia.ceph.com
2022-07-09T06:00:24.165 INFO:teuthology.lock.ops:New key found. Updating...
2022-07-09T06:00:24.247 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server
2022-07-09T06:00:24.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com'
2022-07-09T06:00:24.884 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64
2022-07-09T06:00:24.975 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Red Hat Enterprise Linux"
2022-07-09T06:00:24.988 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8.6 (Ootpa)"
2022-07-09T06:00:24.988 INFO:teuthology.orchestra.run.smithi185.stdout:ID="rhel"
2022-07-09T06:00:24.988 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="fedora"
2022-07-09T06:00:24.989 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8.6"
2022-07-09T06:00:24.989 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8"
2022-07-09T06:00:24.990 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-07-09T06:00:24.991 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31"
2022-07-09T06:00:24.991 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-07-09T06:00:24.992 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.redhat.com/"
2022-07-09T06:00:24.992 INFO:teuthology.orchestra.run.smithi185.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-07-09T06:00:25.001 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-07-09T06:00:25.002 INFO:teuthology.orchestra.run.smithi185.stdout:
2022-07-09T06:00:25.003 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-07-09T06:00:25.003 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-07-09T06:00:25.004 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-07-09T06:00:25.004 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-07-09T06:00:25.020 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server
2022-07-09T06:00:25.061 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: set os type and version on server
2022-07-09T06:00:25.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com'
2022-07-09T06:00:25.679 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64
2022-07-09T06:00:25.737 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Red Hat Enterprise Linux"
2022-07-09T06:00:25.737 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="8.6 (Ootpa)"
2022-07-09T06:00:25.738 INFO:teuthology.orchestra.run.smithi203.stdout:ID="rhel"
2022-07-09T06:00:25.738 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="fedora"
2022-07-09T06:00:25.738 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="8.6"
2022-07-09T06:00:25.739 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el8"
2022-07-09T06:00:25.739 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-07-09T06:00:25.739 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31"
2022-07-09T06:00:25.740 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-07-09T06:00:25.740 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.redhat.com/"
2022-07-09T06:00:25.740 INFO:teuthology.orchestra.run.smithi203.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-07-09T06:00:25.741 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-07-09T06:00:25.741 INFO:teuthology.orchestra.run.smithi203.stdout:
2022-07-09T06:00:25.741 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-07-09T06:00:25.742 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-07-09T06:00:25.742 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-07-09T06:00:25.742 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-07-09T06:00:25.744 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2022-07-09T06:00:26.067 INFO:teuthology.dispatcher.supervisor:Running job 6921367
2022-07-09T06:00:26.067 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_10062088f503b43eff3624326bda825b23438f9b/virtualenv/bin/teuthology -v --owner scheduled_ksirivad@teuthology --archive /home/teuthworker/archive/ksirivad-2022-07-09_05:39:52-rados:thrash-erasure-code-main-distro-default-smithi/6921367 --name ksirivad-2022-07-09_05:39:52-rados:thrash-erasure-code-main-distro-default-smithi --description rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/ksirivad-2022-07-09_05:39:52-rados:thrash-erasure-code-main-distro-default-smithi/6921367/orig.config.yaml
2022-07-09T06:00:26.158 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ksirivad-2022-07-09_05:39:52-rados:thrash-erasure-code-main-distro-default-smithi/6921367
2022-07-09T06:00:26.160 INFO:teuthology.dispatcher.supervisor:Job PID: 30405
2022-07-09T06:00:26.161 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-07-09T18:01:30.243 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-07-09T18:01:30.423 INFO:teuthology.kill:Killing Pids: {30405}
2022-07-09T18:01:31.994 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2022-07-09T18:01:31.995 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2022-07-09T18:01:31.995 INFO:teuthology.misc:Compressing logs...
2022-07-09T18:01:31.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com'
2022-07-09T18:01:32.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com'
2022-07-09T18:01:32.563 INFO:teuthology.misc:Compressing logs...
2022-07-09T18:01:33.472 INFO:teuthology.orchestra.run.smithi203.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-07-09T18:01:33.747 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-07-09T18:01:34.371 INFO:teuthology.misc:Compressing logs...
2022-07-09T18:01:41.167 INFO:teuthology.kill:No teuthology processes running
2022-07-09T18:01:42.530 INFO:teuthology.kill:Nuking machines: ['smithi185', 'smithi203']
2022-07-09T18:01:43.725 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_10062088f503b43eff3624326bda825b23438f9b/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-07-09T18:01:43.726 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-07-09T18:01:43.727 INFO:teuthology.kill:2022-07-09 18:01:43,725.725 INFO:teuthology.nuke:targets:
2022-07-09T18:01:43.727 INFO:teuthology.kill:  smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDclS5jtJC/beXYRkwPdreTmGMIIXB0+Cscw5P+TiDwywpePrye3jnzUlq8qvzjXsj2R2OQUxpDTf5pNZ7koI98sfapru2gBXs8n7wYiJQlNiPPRt5kMxkErFi8TFhm7Yxnsw8atssmd8RV1ITQbElBVXV/xlhdStuY61525pxVUXHmEfzDp9B83N/sVC5pn0d37EClAyisXcdTjNSua7jC+To02O/pcSsE/y4p9y+x2j8Wo7hNdKIXSdqG9ZG6c/zjMhbxIHSdUz6F07g+9rj0Tf1T4DyDPWosCnZgPDc/WOzCP/jSk4A/PnX4jTErACkKlLDLvTEleTh6TsJjSXP6QnjHZv4TvI98Jx1bVMZE/4BGNoCLCwYRFZQZkVHsj4E+rrQ9CleyZGlQgu71REpwT7LuRqFFmKhDVHqUd6uxhTDmQ1sQ5JDOWuhwugxw5MC/W4rB2aQdzAYskSkXXinWhpToR5Ymk9hPsxBtsJtN7HwvC2EWkF9V6f62evGwjcE=
2022-07-09T18:01:43.727 INFO:teuthology.kill:  smithi203.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYpx8qkBoSMvXgip2Pud1Kp+vRi9pHcYrDvc7OPY8JjWWZTsrFPmK1cwJ0a6ITCmLHjeq5lin7flanKcP51RQ/C10rcVAqRdVTVzQeiB5WA2ZMmRwbBjNydl/nPVDjOJD2HySv1BFt3bHDWQoQxs59fxPHGq5xdIrEJpNrpkTsV0ePXr8E4rEyA66LZXX1+HwZwUmo4YLsRRE9G0y4JrWxgtYuLmEBFO8nWfpIZraG8TGF8sGDHLd0bACNQTounR5iWJZVhUb1H3Q9c6IyERV4jLnAC5yKgvNp4UPtr07dRmTY5kuLhzhSoO5N8BnLl4dHarUpYtp6y03KfFGEIpAthmkzSJOvhIrusmnk4b9ehlduud5tXjK2dSf/G8/er0LV4xFwN1R47e8stVgoZox3WwsdI8BtkDBQ+CBYSfsmEA+iuQkFHnl4d/JEjK6RW+q/0PpU1aP367eqTDqmTNpNmQAVsgzkHiw68OYwO4VAogkSobH0yU9/lbJVFIggt/8=
2022-07-09T18:01:43.728 INFO:teuthology.kill:2022-07-09 18:01:43,726.726 INFO:teuthology.nuke:Checking targets against current locks
2022-07-09T18:01:43.793 INFO:teuthology.kill:2022-07-09 18:01:43,793.793 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-07-09T18:01:43.817 INFO:teuthology.kill:2022-07-09 18:01:43,816.816 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-07-09T18:01:43.894 INFO:teuthology.kill:2022-07-09 18:01:43,894.894 INFO:teuthology.orchestra.console:Power off smithi185
2022-07-09T18:01:43.945 INFO:teuthology.kill:2022-07-09 18:01:43,945.945 INFO:teuthology.orchestra.console:Power off smithi203
2022-07-09T18:01:52.285 INFO:teuthology.kill:2022-07-09 18:01:52,285.285 INFO:teuthology.orchestra.console:Power off for smithi185 completed
2022-07-09T18:01:52.315 INFO:teuthology.kill:2022-07-09 18:01:52,314.314 INFO:teuthology.orchestra.console:Power off for smithi203 completed
2022-07-09T18:01:52.445 INFO:teuthology.kill:2022-07-09 18:01:52,444.444 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com
2022-07-09T18:01:52.469 INFO:teuthology.kill:2022-07-09 18:01:52,469.469 INFO:teuthology.lock.ops:unlocked: smithi203.front.sepia.ceph.com
2022-07-09T18:03:52.833 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-07-09T18:03:52.965 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-07-09_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6921564
2022-07-09T18:03:52.966 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi203 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-07-09_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6921564