2022-09-21T07:48:02.454 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18
2022-09-21T07:48:02.651 INFO:teuthology.lock.ops:Start node 'smithi130.front.sepia.ceph.com' reimaging
2022-09-21T07:48:02.653 INFO:teuthology.lock.ops:Updating [smithi130.front.sepia.ceph.com]: reset os type and version on server
2022-09-21T07:48:02.654 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server
2022-09-21T07:48:02.682 INFO:teuthology.lock.ops:Node 'smithi130.front.sepia.ceph.com' reimaging is complete
2022-09-21T07:48:02.682 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging
2022-09-21T07:48:02.683 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server
2022-09-21T07:48:02.684 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2022-09-21T07:48:02.715 INFO:teuthology.lock.ops:Node 'smithi169.front.sepia.ceph.com' reimaging is complete
2022-09-21T07:48:02.915 INFO:teuthology.provision.fog.smithi130:Scheduling deploy of rhel 8.6
2022-09-21T07:48:02.947 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of rhel 8.6
2022-09-21T07:48:03.272 INFO:teuthology.orchestra.console:Power off smithi130
2022-09-21T07:48:03.346 INFO:teuthology.orchestra.console:Power off smithi169
2022-09-21T07:48:03.470 INFO:teuthology.orchestra.console:Power off for smithi130 completed
2022-09-21T07:48:03.537 INFO:teuthology.orchestra.console:Power off for smithi169 completed
2022-09-21T07:48:03.573 INFO:teuthology.orchestra.console:Power on smithi130
2022-09-21T07:48:03.639 INFO:teuthology.orchestra.console:Power on smithi169
2022-09-21T07:48:07.849 INFO:teuthology.orchestra.console:Power on for smithi130 completed
2022-09-21T07:48:07.951 INFO:teuthology.provision.fog.smithi130:Waiting for deploy to finish
2022-09-21T07:48:12.032 INFO:teuthology.orchestra.console:Power on for smithi169 completed
2022-09-21T07:48:12.134 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish
2022-09-21T07:52:46.479 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-21T07:52:46.727 INFO:teuthology.provision.fog.smithi130:Node is ready
2022-09-21T07:52:46.799 INFO:teuthology.orchestra.run.smithi130.stdout:smithi130
2022-09-21T07:52:46.862 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130 smithi130.front.sepia.ceph.com smithi130
2022-09-21T07:52:47.239 INFO:teuthology.provision.fog.smithi130:Deploy complete!
2022-09-21T07:52:56.320 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-21T07:52:56.559 INFO:teuthology.provision.fog.smithi169:Node is ready
2022-09-21T07:52:56.628 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com
2022-09-21T07:52:56.686 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169
2022-09-21T07:52:57.063 INFO:teuthology.provision.fog.smithi169:Deploy complete!
2022-09-21T07:52:57.452 INFO:teuthology.lock.ops:Checking smithi130.front.sepia.ceph.com
2022-09-21T07:52:57.454 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-21T07:52:57.486 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com
2022-09-21T07:52:57.487 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-21T07:52:57.523 INFO:teuthology.lock.ops:Updating [smithi130.front.sepia.ceph.com]: set os type and version on server
2022-09-21T07:52:57.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com'
2022-09-21T07:52:58.014 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64
2022-09-21T07:52:58.080 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="Red Hat Enterprise Linux"
2022-09-21T07:52:58.081 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="8.6 (Ootpa)"
2022-09-21T07:52:58.081 INFO:teuthology.orchestra.run.smithi130.stdout:ID="rhel"
2022-09-21T07:52:58.082 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="fedora"
2022-09-21T07:52:58.082 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="8.6"
2022-09-21T07:52:58.083 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el8"
2022-09-21T07:52:58.084 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-09-21T07:52:58.084 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31"
2022-09-21T07:52:58.085 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-09-21T07:52:58.085 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://www.redhat.com/"
2022-09-21T07:52:58.085 INFO:teuthology.orchestra.run.smithi130.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-09-21T07:52:58.091 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-09-21T07:52:58.092 INFO:teuthology.orchestra.run.smithi130.stdout:
2022-09-21T07:52:58.092 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-09-21T07:52:58.093 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-09-21T07:52:58.093 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-09-21T07:52:58.094 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-09-21T07:52:58.096 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server
2022-09-21T07:52:58.137 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server
2022-09-21T07:52:58.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2022-09-21T07:52:58.461 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64
2022-09-21T07:52:58.548 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Red Hat Enterprise Linux"
2022-09-21T07:52:58.548 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8.6 (Ootpa)"
2022-09-21T07:52:58.549 INFO:teuthology.orchestra.run.smithi169.stdout:ID="rhel"
2022-09-21T07:52:58.549 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="fedora"
2022-09-21T07:52:58.549 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8.6"
2022-09-21T07:52:58.549 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8"
2022-09-21T07:52:58.550 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-09-21T07:52:58.550 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2022-09-21T07:52:58.550 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-09-21T07:52:58.550 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.redhat.com/"
2022-09-21T07:52:58.551 INFO:teuthology.orchestra.run.smithi169.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-09-21T07:52:58.551 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-09-21T07:52:58.551 INFO:teuthology.orchestra.run.smithi169.stdout:
2022-09-21T07:52:58.551 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-09-21T07:52:58.552 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-09-21T07:52:58.552 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-09-21T07:52:58.552 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-09-21T07:52:58.553 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2022-09-21T07:52:58.710 INFO:teuthology.dispatcher.supervisor:Running job 7039861
2022-09-21T07:52:58.711 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_rfriedma@teuthology --archive /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039861 --name rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi --description rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect} -- /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039861/orig.config.yaml
2022-09-21T07:52:58.778 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039861
2022-09-21T07:52:58.778 INFO:teuthology.dispatcher.supervisor:Job PID: 3910
2022-09-21T07:52:58.779 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-09-21T19:53:57.876 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-09-21T19:53:58.216 INFO:teuthology.kill:Killing Pids: {3910}
2022-09-21T19:54:02.253 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2022-09-21T19:54:02.254 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2022-09-21T19:54:02.255 INFO:teuthology.misc:Compressing logs...
2022-09-21T19:54:02.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com'
2022-09-21T19:54:02.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2022-09-21T19:54:03.862 INFO:teuthology.misc:Compressing logs...
2022-09-21T19:54:04.947 INFO:teuthology.orchestra.run.smithi130.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-09-21T19:54:04.949 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-09-21T20:00:36.911 INFO:teuthology.misc:Compressing logs...
2022-09-21T20:00:48.171 INFO:teuthology.kill:No teuthology processes running
2022-09-21T20:00:50.136 INFO:teuthology.kill:Nuking machines: ['smithi130', 'smithi169']
2022-09-21T20:00:51.448 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-09-21T20:00:51.449 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-09-21T20:00:51.449 INFO:teuthology.kill:2022-09-21 20:00:51,447.447 INFO:teuthology.nuke:targets:
2022-09-21T20:00:51.450 INFO:teuthology.kill:  smithi130.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb9yixG5ynSfSDmIpOaiZb5t0Edy3/wPGGfgsyhOWvGfrpJmuUNJ8GNAVKylY2f8Unr23OqrjZMDioHIR5CBUKrh9ApJGZ6GaVhf3YAY+c98GaV0Kd3eG19FsqV8H5HQDudp/N18jl8mO2Yk/eE6yfU3QU5/+TagTIpidtgS5w/G8OqRtXrRs8dwgnWFG5LBz+RWoOXVt7AC3/cXR3WpTT0+8IfZpdiUHaUh05CiP7bw8NjoXOc8U4sRFnogh2ymoc9cj41diSxFLRPTvw3g2kD1hEWKh1sYWCCeusdyTkW+rbjPL+z2O/sK97BWpGtCKWRcmEV9a5YxXlVaLIvMNdEw9f15mRD6oTsOM3VsjJ8VXd7Itg32NJNA6VAvsd59xqdtyKJjshV3J3SjBjrZh899LvqKX9MmO+JtaHzTUEgkqeh0auPcbXdTLiRbDWmYFfmJ1br8mRz/GuCwhsT2+HQZ/kWSWOlcS5M+PBmMASkj4KKB6t+029n7hVQOG8pl8=
2022-09-21T20:00:51.452 INFO:teuthology.kill:  smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFGqBLi0C8/Y6Yb9JQf+ankla3Ynn5+qptN29dyu1qPu1ddUWiiAJ+vC6XxB3TD3duFAkOfg/dW7xt8GU9DIHnWHFkHXw3y+NCTfwBpog4KNQyJp8sx7QFpw507Au5SojbtV9GQc3DPS6ZemCOLOhsHzaEk3f7caZ1cTp4Vdt+C3/KAI6uHqfCbWl9QTYcZkgP4jQqPw8aR4c8U/9R3bbvOvoJehSFsG7rHkQ9v81tM4JEPxWSt7BcJrKhJBIsA0pB/a3avmMAVNTfhZWOluuDYlvDs+cmZ1CQa21ekq/FrV6Rl5a+UES7mINDb0SKQ/MKtxkhSNC9DS8KDte1HLNX8P9NoLNxkhXBA5+B8xd+YCNSCWL3sUXIEdDlH2O9JKCBdGCBlFbhrNkqFBJlRwDh8yV1Ox4Jo5Uzed1xaaPhYTaKKqkIoDoNe3T14Wn/W9ALe1vYSwrXIkQZxV5zShcXpLMmt9KrnpsduMMarv6tJD2Ts0xwhi4Iy7tgsz+xZX8=
2022-09-21T20:00:51.453 INFO:teuthology.kill:2022-09-21 20:00:51,448.448 INFO:teuthology.nuke:Checking targets against current locks
2022-09-21T20:00:51.506 INFO:teuthology.kill:2022-09-21 20:00:51,506.506 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-21T20:00:51.542 INFO:teuthology.kill:2022-09-21 20:00:51,542.542 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-21T20:00:51.596 INFO:teuthology.kill:2022-09-21 20:00:51,595.595 INFO:teuthology.orchestra.console:Power off smithi130
2022-09-21T20:00:51.646 INFO:teuthology.kill:2022-09-21 20:00:51,646.646 INFO:teuthology.orchestra.console:Power off smithi169
2022-09-21T20:00:59.986 INFO:teuthology.kill:2022-09-21 20:00:59,985.985 INFO:teuthology.orchestra.console:Power off for smithi130 completed
2022-09-21T20:01:00.140 INFO:teuthology.kill:2022-09-21 20:01:00,139.139 INFO:teuthology.lock.ops:unlocked: smithi130.front.sepia.ceph.com
2022-09-21T20:01:04.134 INFO:teuthology.kill:2022-09-21 20:01:04,133.133 INFO:teuthology.orchestra.console:Power off for smithi169 completed
2022-09-21T20:01:04.285 INFO:teuthology.kill:2022-09-21 20:01:04,285.285 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com
2022-09-21T20:03:04.646 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-09-21T20:03:04.768 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi130 but it was locked by another job: /home/teuthworker/archive/ivancich-2022-09-21_19:15:04-rgw-wip-eric-testing-1-distro-default-smithi/7040377
2022-09-21T20:03:04.769 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi169 but it was locked by another job: /home/teuthworker/archive/ivancich-2022-09-21_19:15:04-rgw-wip-eric-testing-1-distro-default-smithi/7040377