2022-09-21T07:18:15.900 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-21T07:18:16.058 INFO:teuthology.lock.ops:Start node 'smithi053.front.sepia.ceph.com' reimaging 2022-09-21T07:18:16.059 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: reset os type and version on server 2022-09-21T07:18:16.060 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2022-09-21T07:18:16.083 INFO:teuthology.lock.ops:Node 'smithi053.front.sepia.ceph.com' reimaging is complete 2022-09-21T07:18:16.083 INFO:teuthology.lock.ops:Start node 'smithi102.front.sepia.ceph.com' reimaging 2022-09-21T07:18:16.084 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: reset os type and version on server 2022-09-21T07:18:16.085 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2022-09-21T07:18:16.106 INFO:teuthology.lock.ops:Node 'smithi102.front.sepia.ceph.com' reimaging is complete 2022-09-21T07:18:16.312 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of centos 8.stream 2022-09-21T07:18:16.382 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of centos 8.stream 2022-09-21T07:18:16.789 INFO:teuthology.orchestra.console:Power off smithi053 2022-09-21T07:18:16.868 INFO:teuthology.orchestra.console:Power off smithi102 2022-09-21T07:18:25.166 INFO:teuthology.orchestra.console:Power off for smithi053 completed 2022-09-21T07:18:25.268 INFO:teuthology.orchestra.console:Power on smithi053 2022-09-21T07:18:29.345 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2022-09-21T07:18:29.448 INFO:teuthology.orchestra.console:Power on smithi102 2022-09-21T07:18:33.638 INFO:teuthology.orchestra.console:Power on for smithi053 completed 2022-09-21T07:18:33.702 INFO:teuthology.orchestra.console:Power on for smithi102 completed 2022-09-21T07:18:33.740 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish 2022-09-21T07:18:33.803 INFO:teuthology.provision.fog.smithi102:Waiting for deploy to finish 2022-09-21T07:22:41.862 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-21T07:22:42.179 INFO:teuthology.provision.fog.smithi053:Node is ready 2022-09-21T07:22:42.201 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053.front.sepia.ceph.com 2022-09-21T07:22:42.261 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53 smithi053.front.sepia.ceph.com smithi053 2022-09-21T07:22:42.633 INFO:teuthology.provision.fog.smithi053:Deploy complete! 2022-09-21T07:23:21.088 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-21T07:23:21.401 INFO:teuthology.provision.fog.smithi102:Node is ready 2022-09-21T07:23:21.435 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102.front.sepia.ceph.com 2022-09-21T07:23:21.502 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102 smithi102.front.sepia.ceph.com smithi102 2022-09-21T07:23:21.865 INFO:teuthology.provision.fog.smithi102:Deploy complete! 2022-09-21T07:23:22.208 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com 2022-09-21T07:23:22.210 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-21T07:23:22.244 INFO:teuthology.lock.ops:Checking smithi102.front.sepia.ceph.com 2022-09-21T07:23:22.245 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-21T07:23:22.276 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server 2022-09-21T07:23:22.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2022-09-21T07:23:22.675 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2022-09-21T07:23:22.735 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2022-09-21T07:23:22.735 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8" 2022-09-21T07:23:22.735 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2022-09-21T07:23:22.735 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2022-09-21T07:23:22.736 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8" 2022-09-21T07:23:22.736 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2022-09-21T07:23:22.736 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-21T07:23:22.736 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2022-09-21T07:23:22.737 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-21T07:23:22.737 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2022-09-21T07:23:22.737 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-21T07:23:22.737 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-21T07:23:22.738 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-21T07:23:22.739 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2022-09-21T07:23:22.762 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: set os type and version on server 2022-09-21T07:23:22.764 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2022-09-21T07:23:23.031 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2022-09-21T07:23:23.091 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2022-09-21T07:23:23.091 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8" 2022-09-21T07:23:23.091 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2022-09-21T07:23:23.092 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2022-09-21T07:23:23.092 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8" 2022-09-21T07:23:23.092 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8" 2022-09-21T07:23:23.092 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-21T07:23:23.093 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2022-09-21T07:23:23.093 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-21T07:23:23.093 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2022-09-21T07:23:23.093 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-21T07:23:23.094 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-21T07:23:23.094 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-21T07:23:23.094 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2022-09-21T07:23:23.351 INFO:teuthology.dispatcher.supervisor:Running job 7039826 2022-09-21T07:23:23.352 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_rfriedma@teuthology --archive /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039826 --name rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi --description rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} -- /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039826/orig.config.yaml 2022-09-21T07:23:23.377 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039826 2022-09-21T07:23:23.378 INFO:teuthology.dispatcher.supervisor:Job PID: 5047 2022-09-21T07:23:23.379 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-21T19:24:22.064 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-21T19:24:22.285 INFO:teuthology.kill:Killing Pids: {5047} 2022-09-21T19:24:24.784 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-21T19:24:24.786 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-21T19:24:24.786 INFO:teuthology.misc:Compressing logs... 2022-09-21T19:24:24.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2022-09-21T19:24:25.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2022-09-21T19:24:25.597 INFO:teuthology.misc:Compressing logs... 2022-09-21T19:24:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-21T19:24:26.894 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-21T19:30:25.337 INFO:teuthology.misc:Compressing logs... 2022-09-21T19:30:35.370 INFO:teuthology.kill:No teuthology processes running 2022-09-21T19:30:37.418 INFO:teuthology.kill:Nuking machines: ['smithi053', 'smithi102'] 2022-09-21T19:30:39.046 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-21T19:30:39.047 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-21T19:30:39.047 INFO:teuthology.kill:2022-09-21 19:30:39,034.034 INFO:teuthology.nuke:targets: 2022-09-21T19:30:39.050 INFO:teuthology.kill: smithi053.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC18ClKlhnUwAI5QoEIREJ2HWzJusrWhboChx6wrD8DC9WB2jJ+fS+aHAuOLpMp7J/+ykfypjr2Zyi8pohJzvtBtrfjIMJS5SZlPIy7tnTZkq8AkrhHU8DWVe7wRDV13UvkkbNVEp1VChyBvCsYsz0yj1C1xZBp+xpFH0sqHMpaNpw5LDuNF4u7hBgL5eyJhg8JWFDhFaO/rqI2CVRh1Om1YURLV3914/hiJVYyyrdPniiFdt+ZkEZ1TvIyV1cclcDjVyZwL/OIZLm8YkgxEXu/5SwBvg4JYFCNycD5JwYPSjHbeIVTg8wp/TUGCnuJudQhdVB7jicyPdk8lUrlRokJDRNSsUCV0i00XQNgYdknFzl7FOF+hBtFIkHp47wyDX2g1AXDnEYdKxuNJq016bCK1bTyedJ2vRt1EfHO77I8YN/z1BlzpoB5wfPUREPDFaDXEyo/JDK0OelQLoXcHy5mu/5ruXIH5rwMhRtdh9WUG1XfkE5vPfwnPjEX1tQRNZ0= 2022-09-21T19:30:39.050 INFO:teuthology.kill: smithi102.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRKon+Dfz++nfEvssubQL7NmsUUni0mn+/86AqR1dSWTD8e/na/3XM8adnuTQiX5hxsRtbOF1HC7y+ZrpnEXNQrQz3qI390D4ni62uBMzRSX8u8Q/OACWE6s4+AD8Rn0KA4F764FSsxCit3q3t/WFufn7jSMo9or13DugTqpV62JyzzbhVEK+1hY8wpa2s0MOaOCZXAvCR4kacRgIIh+Yl07pPYV5XMZWcKytfWpWIZrX+4WlQcsoqsTgIGAKMSOFplWJzB7VmKRIebCjMJEhVS46evKX3JH7AwhOWjUQ1MVqFFUlhANZmvkN1bsndcn36J33FEpSAR3Ix71u4E8aRFTuUepUpTXrEM0NO/rLZRfZU3BQu8Mt2V/uxn1aoiZiCD/LiXh867DkHGmZbqkkcIRsHkw4Wv83dSpkP5bcRJxT508iyv5qKOk1FdpYdX+Nit8FSE01E5fR59gq9AwL05Z07b0tA6LeQ7QzD1CAwknRQnmA8UTpNZlU8jLcMPTc= 2022-09-21T19:30:39.051 INFO:teuthology.kill:2022-09-21 19:30:39,047.047 INFO:teuthology.nuke:Checking targets against current locks 2022-09-21T19:30:39.110 INFO:teuthology.kill:2022-09-21 19:30:39,109.109 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-21T19:30:39.138 INFO:teuthology.kill:2022-09-21 19:30:39,137.137 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-21T19:30:39.206 INFO:teuthology.kill:2022-09-21 19:30:39,206.206 INFO:teuthology.orchestra.console:Power off smithi053 2022-09-21T19:30:39.266 INFO:teuthology.kill:2022-09-21 19:30:39,265.265 INFO:teuthology.orchestra.console:Power off smithi102 2022-09-21T19:30:47.620 INFO:teuthology.kill:2022-09-21 19:30:47,619.619 INFO:teuthology.orchestra.console:Power off for smithi053 completed 2022-09-21T19:30:47.677 INFO:teuthology.kill:2022-09-21 19:30:47,677.677 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2022-09-21T19:30:47.800 INFO:teuthology.kill:2022-09-21 19:30:47,799.799 INFO:teuthology.lock.ops:unlocked: smithi053.front.sepia.ceph.com 2022-09-21T19:30:47.830 INFO:teuthology.kill:2022-09-21 19:30:47,830.830 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com 2022-09-21T19:32:48.315 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-21T19:32:48.465 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi053 but it was locked by another job: /home/teuthworker/archive/ivancich-2022-09-21_19:14:37-rgw-wip-eric-testing-1-distro-default-smithi/7040339 2022-09-21T19:32:48.466 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi102 but it was locked by another job: /home/teuthworker/archive/ivancich-2022-09-21_19:14:37-rgw-wip-eric-testing-1-distro-default-smithi/7040339