2022-02-10T07:59:10.934 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-10T07:59:11.061 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi037.front.sepia.ceph.com' reimaging 2022-02-10T07:59:11.062 INFO:teuthology.lock.ops:Updating [ubuntu@smithi037.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T07:59:11.063 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2022-02-10T07:59:11.101 INFO:teuthology.lock.ops:Node 'ubuntu@smithi037.front.sepia.ceph.com' reimaging is complete 2022-02-10T07:59:11.101 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging 2022-02-10T07:59:11.102 INFO:teuthology.lock.ops:Updating [ubuntu@smithi150.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T07:59:11.102 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2022-02-10T07:59:11.125 INFO:teuthology.lock.ops:Node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging is complete 2022-02-10T07:59:11.327 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of centos 8.stream 2022-02-10T07:59:11.337 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of centos 8.stream 2022-02-10T07:59:11.774 INFO:teuthology.orchestra.console:Power off smithi037 2022-02-10T07:59:11.789 INFO:teuthology.orchestra.console:Power off smithi150 2022-02-10T07:59:11.919 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2022-02-10T07:59:11.928 INFO:teuthology.orchestra.console:Power off for smithi150 completed 2022-02-10T07:59:12.020 INFO:teuthology.orchestra.console:Power on smithi037 2022-02-10T07:59:12.029 INFO:teuthology.orchestra.console:Power on smithi150 2022-02-10T07:59:16.289 INFO:teuthology.orchestra.console:Power on for smithi150 completed 2022-02-10T07:59:16.390 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish 2022-02-10T07:59:20.395 INFO:teuthology.orchestra.console:Power on for smithi037 completed 2022-02-10T07:59:20.496 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish 2022-02-10T08:02:54.503 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T08:03:14.788 INFO:teuthology.provision.fog.smithi037:Node is ready 2022-02-10T08:03:14.813 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com 2022-02-10T08:03:14.873 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037 2022-02-10T08:03:15.274 INFO:teuthology.provision.fog.smithi037:Deploy complete! 2022-02-10T08:04:13.398 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T08:04:13.671 INFO:teuthology.provision.fog.smithi150:Node is ready 2022-02-10T08:04:13.688 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com 2022-02-10T08:04:13.746 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150 2022-02-10T08:04:14.096 INFO:teuthology.provision.fog.smithi150:Deploy complete! 2022-02-10T08:04:14.255 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com 2022-02-10T08:04:14.256 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T08:04:14.290 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com 2022-02-10T08:04:14.291 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T08:04:14.317 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server 2022-02-10T08:04:14.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2022-02-10T08:04:14.534 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2022-02-10T08:04:14.599 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2022-02-10T08:04:14.600 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8" 2022-02-10T08:04:14.600 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2022-02-10T08:04:14.601 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2022-02-10T08:04:14.601 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8" 2022-02-10T08:04:14.603 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8" 2022-02-10T08:04:14.604 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-10T08:04:14.604 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2022-02-10T08:04:14.605 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-10T08:04:14.605 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2022-02-10T08:04:14.606 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-10T08:04:14.606 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-10T08:04:14.607 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-10T08:04:14.609 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2022-02-10T08:04:14.642 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server 2022-02-10T08:04:14.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2022-02-10T08:04:14.845 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2022-02-10T08:04:14.904 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2022-02-10T08:04:14.905 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8" 2022-02-10T08:04:14.905 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2022-02-10T08:04:14.906 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2022-02-10T08:04:14.906 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8" 2022-02-10T08:04:14.907 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2022-02-10T08:04:14.907 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-10T08:04:14.908 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2022-02-10T08:04:14.908 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-10T08:04:14.908 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2022-02-10T08:04:14.909 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-10T08:04:14.909 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-10T08:04:14.910 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-10T08:04:14.911 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2022-02-10T08:04:15.016 INFO:teuthology.dispatcher.supervisor:Running job 6673160 2022-02-10T08:04:15.016 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_amathuri@teuthology --archive /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673160 --name amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673160/orig.config.yaml 2022-02-10T08:04:15.020 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673160 2022-02-10T08:04:15.021 INFO:teuthology.dispatcher.supervisor:Job PID: 25871 2022-02-10T08:04:15.022 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-10T14:34:25.753 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-10T14:34:25.949 INFO:teuthology.kill:Killing Pids: {25871} 2022-02-10T14:34:27.057 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-10T14:34:27.057 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-10T14:34:27.058 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:34:27.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2022-02-10T14:34:27.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2022-02-10T14:34:27.563 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:34:28.206 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:34:58.662 INFO:teuthology.kill:No teuthology processes running 2022-02-10T14:34:59.802 INFO:teuthology.kill:Nuking machines: ['smithi037', 'smithi150'] 2022-02-10T14:35:00.771 INFO:teuthology.kill:2022-02-10 14:35:00,771.771 INFO:teuthology.nuke:targets: 2022-02-10T14:35:00.772 INFO:teuthology.kill: smithi037.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC11Wmqs2eWuuiqz6qzqQ4REKZw7b0R9fUEF9f/SrrazS4iiTbvYbA8JxusIfnNe+Su6cKWBdISC+NnsMDCDF5QO3veTmtEbaB/HXwsfFMZDho29Z7lw8BzKfCuuQNyD6tqXhI7bsmrNJrYAYuXSs00TNKzlKmEB7T1Z90lBUE+OSg8MmTVk6Gf3s3g+BgadQA/F0RiVRWG/XnyggWsONXJuRQ1pjKeHOB21IXVVgBV8E6tu2lo91u4QvBoOPrdGeCHcnOPVuupdCMumk6QCeou2ZFP2blFWjt0M/gK4boTptXZCF1QcRAX/ufO8VCREJ0y1+RD0HNXpsd3ShOmJGPHoLvUiFtHAW2uwYSU+exKd+p2YRFwvmYYXZu9frUA9XcALC856Ans/docF8UZ1PnKogHsC4NTU+qbXJdmlBb5ni3ZPkTRIVncdooumf9/AOB1PGPHnfJjLjrgY/zL+XyBijJVQCXhR6CqI2GZpWMUWFBFci/fC3iRY0smCJEMWA0= 2022-02-10T14:35:00.772 INFO:teuthology.kill: smithi150.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfzhlp597z2BafLn1yYOhfa2E576o0w2t62WoNiis4dwkk7Y/ui8LCe+9BlDwvEDvcZD3npGaXHzRjtg3hfNAEceD5llOheHtMs03+K+EY7fXWuqsFgmcAMn4EVbq5xNODUrKeTIjy6QNDcDV4n2I6714T9Oq4T4xC/8hq6PzzOG1LjljM8b5Nbb6chpfDq/IrCcKvyKoczAb8Ms/3bJuMSBopuwvOXiZP/TDzjQQ1wkl/h1FY4ZM4PQeA1fjKGIU5n0oDioC+UoLqmPp8+NoKPs90OhCIuoYeR5VzginPyHPUgdmQgT3hfITnykAoRTRPCxgV14Gw8fnJavM7qWctVBomP4LUJlhLRsMyQ4ihARIWKAhP5Kz/S3vWTZgG+xX2h4+Q2NwPquR0bi+p2+0c1vqkJzPNv2uPONipKXNr4bi3/9MQjqlGfpZVQN4BPRJdm3jHGnjKt/piW9d1dujQFsrudDksN305bIUHeJc4GtXGBVWVFpUaYWtfKx/d5EE= 2022-02-10T14:35:00.799 INFO:teuthology.kill:2022-02-10 14:35:00,799.799 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T14:35:00.802 INFO:teuthology.kill:2022-02-10 14:35:00,802.802 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T14:35:00.879 INFO:teuthology.kill:2022-02-10 14:35:00,879.879 INFO:teuthology.orchestra.console:Power off smithi037 2022-02-10T14:35:00.892 INFO:teuthology.kill:2022-02-10 14:35:00,892.892 INFO:teuthology.orchestra.console:Power off smithi150 2022-02-10T14:35:09.265 INFO:teuthology.kill:2022-02-10 14:35:09,264.264 INFO:teuthology.orchestra.console:Power off for smithi150 completed 2022-02-10T14:35:09.410 INFO:teuthology.kill:2022-02-10 14:35:09,409.409 INFO:teuthology.lock.ops:unlocked: smithi150.front.sepia.ceph.com 2022-02-10T14:35:13.368 INFO:teuthology.kill:2022-02-10 14:35:13,367.367 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2022-02-10T14:35:13.514 INFO:teuthology.kill:2022-02-10 14:35:13,513.513 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com 2022-02-10T14:37:13.825 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-10T14:37:13.932 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi037 but it was locked by another job: /home/teuthworker/archive/matan-2022-02-10_14:11:48-rados:thrash-master-distro-basic-smithi/6673705 2022-02-10T14:37:13.933 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi150 but it was locked by another job: /home/teuthworker/archive/matan-2022-02-10_14:11:48-rados:thrash-master-distro-basic-smithi/6673705