2022-09-21T07:41:40.980 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-21T07:41:41.214 INFO:teuthology.lock.ops:Start node 'smithi061.front.sepia.ceph.com' reimaging 2022-09-21T07:41:41.218 INFO:teuthology.lock.ops:Updating [smithi061.front.sepia.ceph.com]: reset os type and version on server 2022-09-21T07:41:41.218 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2022-09-21T07:41:41.243 INFO:teuthology.lock.ops:Node 'smithi061.front.sepia.ceph.com' reimaging is complete 2022-09-21T07:41:41.244 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging 2022-09-21T07:41:41.244 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server 2022-09-21T07:41:41.244 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2022-09-21T07:41:41.292 INFO:teuthology.lock.ops:Node 'smithi062.front.sepia.ceph.com' reimaging is complete 2022-09-21T07:41:41.475 INFO:teuthology.provision.fog.smithi061:Scheduling deploy of ubuntu 20.04 2022-09-21T07:41:41.531 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of ubuntu 20.04 2022-09-21T07:41:41.835 INFO:teuthology.orchestra.console:Power off smithi061 2022-09-21T07:41:42.018 INFO:teuthology.orchestra.console:Power off smithi062 2022-09-21T07:41:50.254 INFO:teuthology.orchestra.console:Power off for smithi061 completed 2022-09-21T07:41:50.356 INFO:teuthology.orchestra.console:Power on smithi061 2022-09-21T07:41:54.590 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2022-09-21T07:41:54.692 INFO:teuthology.orchestra.console:Power on smithi062 2022-09-21T07:41:58.747 INFO:teuthology.orchestra.console:Power on for smithi061 completed 2022-09-21T07:41:58.849 INFO:teuthology.provision.fog.smithi061:Waiting for deploy to finish 2022-09-21T07:41:58.979 INFO:teuthology.orchestra.console:Power on for smithi062 completed 2022-09-21T07:41:59.083 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish 2022-09-21T07:47:02.459 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-21T07:47:03.016 INFO:teuthology.provision.fog.smithi062:Node is ready 2022-09-21T07:47:03.064 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com 2022-09-21T07:47:03.075 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062 2022-09-21T07:47:03.252 INFO:teuthology.provision.fog.smithi062:Deploy complete! 2022-09-21T07:47:08.437 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-21T07:47:08.972 INFO:teuthology.provision.fog.smithi061:Node is ready 2022-09-21T07:47:08.978 INFO:teuthology.orchestra.run.smithi061.stdout:smithi061.front.sepia.ceph.com 2022-09-21T07:47:09.028 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61 smithi061.front.sepia.ceph.com smithi061 2022-09-21T07:47:09.200 INFO:teuthology.provision.fog.smithi061:Deploy complete! 2022-09-21T07:47:09.447 INFO:teuthology.lock.ops:Checking smithi061.front.sepia.ceph.com 2022-09-21T07:47:09.448 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-21T07:47:09.468 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com 2022-09-21T07:47:09.469 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-21T07:47:09.488 INFO:teuthology.lock.ops:Updating [smithi061.front.sepia.ceph.com]: set os type and version on server 2022-09-21T07:47:09.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2022-09-21T07:47:09.768 INFO:teuthology.orchestra.run.smithi061.stdout:x86_64 2022-09-21T07:47:09.775 INFO:teuthology.orchestra.run.smithi061.stdout:NAME="Ubuntu" 2022-09-21T07:47:09.776 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-09-21T07:47:09.776 INFO:teuthology.orchestra.run.smithi061.stdout:ID=ubuntu 2022-09-21T07:47:09.776 INFO:teuthology.orchestra.run.smithi061.stdout:ID_LIKE=debian 2022-09-21T07:47:09.776 INFO:teuthology.orchestra.run.smithi061.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-09-21T07:47:09.777 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_ID="20.04" 2022-09-21T07:47:09.777 INFO:teuthology.orchestra.run.smithi061.stdout:HOME_URL="https://www.ubuntu.com/" 2022-09-21T07:47:09.777 INFO:teuthology.orchestra.run.smithi061.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-09-21T07:47:09.777 INFO:teuthology.orchestra.run.smithi061.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-09-21T07:47:09.778 INFO:teuthology.orchestra.run.smithi061.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-09-21T07:47:09.778 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_CODENAME=focal 2022-09-21T07:47:09.779 INFO:teuthology.orchestra.run.smithi061.stdout:UBUNTU_CODENAME=focal 2022-09-21T07:47:09.780 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2022-09-21T07:47:09.802 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server 2022-09-21T07:47:09.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2022-09-21T07:47:10.052 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2022-09-21T07:47:10.058 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Ubuntu" 2022-09-21T07:47:10.058 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-09-21T07:47:10.059 INFO:teuthology.orchestra.run.smithi062.stdout:ID=ubuntu 2022-09-21T07:47:10.059 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE=debian 2022-09-21T07:47:10.059 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-09-21T07:47:10.059 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="20.04" 2022-09-21T07:47:10.060 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.ubuntu.com/" 2022-09-21T07:47:10.061 INFO:teuthology.orchestra.run.smithi062.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-09-21T07:47:10.061 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-09-21T07:47:10.061 INFO:teuthology.orchestra.run.smithi062.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-09-21T07:47:10.061 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_CODENAME=focal 2022-09-21T07:47:10.062 INFO:teuthology.orchestra.run.smithi062.stdout:UBUNTU_CODENAME=focal 2022-09-21T07:47:10.062 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2022-09-21T07:47:10.186 INFO:teuthology.dispatcher.supervisor:Running job 7039855 2022-09-21T07:47:10.186 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_rfriedma@teuthology --archive /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039855 --name rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi --description rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} -- /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039855/orig.config.yaml 2022-09-21T07:47:10.194 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rfriedma-2022-09-21_06:47:34-rados:thrash-wip-rf-tout-distro-default-smithi/7039855 2022-09-21T07:47:10.195 INFO:teuthology.dispatcher.supervisor:Job PID: 17091 2022-09-21T07:47:10.196 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-21T19:48:09.176 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-21T19:48:10.218 INFO:teuthology.kill:Killing Pids: {17091} 2022-09-21T19:48:12.861 INFO:teuthology.task.internal:roles: ubuntu@smithi061.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-21T19:48:12.862 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-21T19:48:12.863 INFO:teuthology.misc:Compressing logs... 2022-09-21T19:48:12.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2022-09-21T19:48:13.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2022-09-21T19:48:13.880 INFO:teuthology.misc:Compressing logs... 2022-09-21T19:57:14.768 INFO:teuthology.misc:Compressing logs... 2022-09-21T19:57:28.239 INFO:teuthology.kill:No teuthology processes running 2022-09-21T19:57:31.536 INFO:teuthology.kill:Nuking machines: ['smithi061', 'smithi062'] 2022-09-21T19:57:35.043 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-21T19:57:35.045 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-21T19:57:35.045 INFO:teuthology.kill:2022-09-21 19:57:35,042.042 INFO:teuthology.nuke:targets: 2022-09-21T19:57:35.046 INFO:teuthology.kill: smithi061.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDbaWEgOpwzNNhP/jo0acQDeagjOCTXW3a0QDbfnKeGPYM42piBCp8c52gFYONcSLnAxki6DzDH5SoFiGFHr3U5FSTQIjd7JsaZ7W/frXs74t9RJ4CojC5Cz3Gg1rHSSkhGQo59zR6CvfO4djDXHWwDz4t3bjCfOg6CI0NqN2gPnXsaQW0ARp6t+wANg6eGo878kI/p2HchXLbUackJBRJ2UpdIi3prcyqULAzEJ3MaNcSG4YKMJHbCOBFxosj520d1SDBqsA6UQNyPIhXnEJMgHsF4oUqkb5LqsbVemYXGY0QQsjEFqz9wiCF+9YjddAprf6k+6GUlNRrQ1IGC5NfQiR0Q+/XouwR845sp0C9VeoFN0o8PEGC1+ffoVL8lDHNyFkwVH3N2DVWRKoRNg11YwCTVFUolfZ4+UNOApjLxIvTDDo8uzqTu+8Z+MmJ09unJJizTUuDjO1FX7/cXqFuc8HhL7OnKyK/XYcW4xK+VSzB8WwcfTQkUS+3soHHrUc= 2022-09-21T19:57:35.047 INFO:teuthology.kill: smithi062.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMXE4qUen++RrdDJcJY2HXFjrgfnpIVj1L9WJCcHoTH/pbG9mhHcLSnuZZHRvln4tIiZ+fNVBYLPCowN+t4LviCSq6E/PMvEwONatj/Tch8WQgRpaX8K89rM32RtOwE2BZIGl/QQiRQzrAR5pUckJbZ5i1jZqebTVOG7LtXLe9MzUU+VzON6e2Wp0Nnj5V7NvV7vNvlE1pSAkHtSQXZBpc9WirRx1XoEKlEY5A2bIPgKzV5nVH/bZ8oAwABibGxaA1Zq7dADNLHZYj2mpuU+kx2h5s7P4cHrbDdL4CkZNKm7Q2ORkULvBFl5/cCqBv1BrHWYpIw7J3QO3/jIf2zGXROabXoIq+C0dvOuSWfjLwHuB6zybAwM8KNzS+LkCj7ECoeJiA77HAZHuntW8VRvgSHMg3xX+9u5YNOOF/dwFaU1UXz4SZPxhciZYHdvyura9QoAPe99jnRaDwrl6LTZiacjR2M55wt+gb04gXY6xpNs73wB4QpbpTybXVV4a43hc= 2022-09-21T19:57:35.057 INFO:teuthology.kill:2022-09-21 19:57:35,057.057 INFO:teuthology.nuke:Checking targets against current locks 2022-09-21T19:57:35.189 INFO:teuthology.kill:2022-09-21 19:57:35,188.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-21T19:57:35.236 INFO:teuthology.kill:2022-09-21 19:57:35,236.236 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-21T19:57:35.370 INFO:teuthology.kill:2022-09-21 19:57:35,370.370 INFO:teuthology.orchestra.console:Power off smithi061 2022-09-21T19:57:35.474 INFO:teuthology.kill:2022-09-21 19:57:35,463.463 INFO:teuthology.orchestra.console:Power off smithi062 2022-09-21T19:57:48.054 INFO:teuthology.kill:2022-09-21 19:57:48,053.053 INFO:teuthology.orchestra.console:Power off for smithi061 completed 2022-09-21T19:57:48.075 INFO:teuthology.kill:2022-09-21 19:57:48,075.075 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2022-09-21T19:57:48.271 INFO:teuthology.kill:2022-09-21 19:57:48,269.269 INFO:teuthology.lock.ops:unlocked: smithi061.front.sepia.ceph.com 2022-09-21T19:57:48.301 INFO:teuthology.kill:2022-09-21 19:57:48,300.300 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com 2022-09-21T19:59:48.828 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-21T19:59:48.986 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi061 but it was locked by another job: /home/teuthworker/archive/ivancich-2022-09-21_19:15:04-rgw-wip-eric-testing-1-distro-default-smithi/7040372 2022-09-21T19:59:48.986 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi062 but it was locked by another job: /home/teuthworker/archive/ivancich-2022-09-21_19:15:04-rgw-wip-eric-testing-1-distro-default-smithi/7040371