2022-01-05T05:10:50.486 INFO:root:teuthology version: 1.1.0-95a7d479 2022-01-05T05:10:50.646 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi039.front.sepia.ceph.com' reimaging 2022-01-05T05:10:50.647 INFO:teuthology.lock.ops:Updating [ubuntu@smithi039.front.sepia.ceph.com]: reset os type and version on server 2022-01-05T05:10:50.647 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2022-01-05T05:10:50.667 INFO:teuthology.lock.ops:Node 'ubuntu@smithi039.front.sepia.ceph.com' reimaging is complete 2022-01-05T05:10:50.667 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi160.front.sepia.ceph.com' reimaging 2022-01-05T05:10:50.667 INFO:teuthology.lock.ops:Updating [ubuntu@smithi160.front.sepia.ceph.com]: reset os type and version on server 2022-01-05T05:10:50.668 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2022-01-05T05:10:50.686 INFO:teuthology.lock.ops:Node 'ubuntu@smithi160.front.sepia.ceph.com' reimaging is complete 2022-01-05T05:10:50.687 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi175.front.sepia.ceph.com' reimaging 2022-01-05T05:10:50.687 INFO:teuthology.lock.ops:Updating [ubuntu@smithi175.front.sepia.ceph.com]: reset os type and version on server 2022-01-05T05:10:50.687 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2022-01-05T05:10:50.706 INFO:teuthology.lock.ops:Node 'ubuntu@smithi175.front.sepia.ceph.com' reimaging is complete 2022-01-05T05:10:50.893 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of centos 8.stream 2022-01-05T05:10:50.896 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of centos 8.stream 2022-01-05T05:10:50.907 INFO:teuthology.provision.fog.smithi175:Scheduling deploy of centos 8.stream 2022-01-05T05:10:51.335 INFO:teuthology.orchestra.console:Power off smithi175 2022-01-05T05:10:51.348 INFO:teuthology.orchestra.console:Power off smithi039 2022-01-05T05:10:51.414 INFO:teuthology.orchestra.console:Power off smithi160 2022-01-05T05:10:59.701 INFO:teuthology.orchestra.console:Power off for smithi175 completed 2022-01-05T05:10:59.710 INFO:teuthology.orchestra.console:Power off for smithi039 completed 2022-01-05T05:10:59.776 INFO:teuthology.orchestra.console:Power off for smithi160 completed 2022-01-05T05:10:59.802 INFO:teuthology.orchestra.console:Power on smithi175 2022-01-05T05:10:59.811 INFO:teuthology.orchestra.console:Power on smithi039 2022-01-05T05:10:59.877 INFO:teuthology.orchestra.console:Power on smithi160 2022-01-05T05:11:08.168 INFO:teuthology.orchestra.console:Power on for smithi175 completed 2022-01-05T05:11:08.179 INFO:teuthology.orchestra.console:Power on for smithi039 completed 2022-01-05T05:11:08.235 INFO:teuthology.orchestra.console:Power on for smithi160 completed 2022-01-05T05:11:08.270 INFO:teuthology.provision.fog.smithi175:Waiting for deploy to finish 2022-01-05T05:11:08.280 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish 2022-01-05T05:11:08.336 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish 2022-01-05T05:15:19.637 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-05T05:15:26.593 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-05T05:15:39.852 INFO:teuthology.provision.fog.smithi175:Node is ready 2022-01-05T05:15:39.873 INFO:teuthology.orchestra.run.smithi175.stdout:smithi126 2022-01-05T05:15:39.931 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.126 smithi126.front.sepia.ceph.com smithi126 2022-01-05T05:15:40.342 INFO:teuthology.provision.fog.smithi175:Deploy complete! 2022-01-05T05:15:52.026 INFO:teuthology.provision.fog.smithi160:Node is ready 2022-01-05T05:15:52.088 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com 2022-01-05T05:15:52.146 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160 2022-01-05T05:15:52.503 INFO:teuthology.provision.fog.smithi160:Deploy complete! 2022-01-05T05:16:43.951 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-05T05:16:44.244 INFO:teuthology.provision.fog.smithi039:Node is ready 2022-01-05T05:16:44.261 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039.front.sepia.ceph.com 2022-01-05T05:16:44.319 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39 smithi039.front.sepia.ceph.com smithi039 2022-01-05T05:16:44.660 INFO:teuthology.provision.fog.smithi039:Deploy complete! 2022-01-05T05:16:44.835 INFO:teuthology.lock.ops:Checking smithi039.front.sepia.ceph.com 2022-01-05T05:16:44.836 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-05T05:16:44.859 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com 2022-01-05T05:16:44.859 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-05T05:16:44.876 INFO:teuthology.lock.ops:Checking smithi175.front.sepia.ceph.com 2022-01-05T05:16:44.877 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-05T05:16:44.895 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: set os type and version on server 2022-01-05T05:16:44.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2022-01-05T05:16:45.091 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2022-01-05T05:16:45.147 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2022-01-05T05:16:45.147 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8" 2022-01-05T05:16:45.148 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2022-01-05T05:16:45.148 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2022-01-05T05:16:45.148 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8" 2022-01-05T05:16:45.148 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2022-01-05T05:16:45.149 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-05T05:16:45.149 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2022-01-05T05:16:45.149 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-05T05:16:45.149 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2022-01-05T05:16:45.150 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-05T05:16:45.150 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-05T05:16:45.150 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-05T05:16:45.151 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2022-01-05T05:16:45.170 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server 2022-01-05T05:16:45.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2022-01-05T05:16:45.414 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2022-01-05T05:16:45.476 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2022-01-05T05:16:45.476 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8" 2022-01-05T05:16:45.476 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2022-01-05T05:16:45.477 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2022-01-05T05:16:45.477 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8" 2022-01-05T05:16:45.477 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2022-01-05T05:16:45.477 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-05T05:16:45.478 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2022-01-05T05:16:45.478 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-05T05:16:45.478 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2022-01-05T05:16:45.478 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-05T05:16:45.479 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-05T05:16:45.479 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-05T05:16:45.479 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2022-01-05T05:16:45.507 INFO:teuthology.lock.ops:Updating [smithi175.front.sepia.ceph.com]: set os type and version on server 2022-01-05T05:16:45.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2022-01-05T05:16:45.753 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2022-01-05T05:16:45.810 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2022-01-05T05:16:45.810 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="8" 2022-01-05T05:16:45.810 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2022-01-05T05:16:45.811 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2022-01-05T05:16:45.811 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="8" 2022-01-05T05:16:45.811 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el8" 2022-01-05T05:16:45.811 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-05T05:16:45.812 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2022-01-05T05:16:45.812 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-05T05:16:45.812 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2022-01-05T05:16:45.812 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-05T05:16:45.813 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-05T05:16:45.813 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-05T05:16:45.813 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2022-01-05T05:16:45.907 INFO:teuthology.dispatcher.supervisor:Running job 6595716 2022-01-05T05:16:45.907 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-04_22:01:43-rbd-wip-yuri7-testing-2022-01-04-1159-distro-default-smithi/6595716 --name yuriw-2022-01-04_22:01:43-rbd-wip-yuri7-testing-2022-01-04-1159-distro-default-smithi --description rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} thrashers/cache thrashosds-health workloads/rbd_fsx_nbd} -- /home/teuthworker/archive/yuriw-2022-01-04_22:01:43-rbd-wip-yuri7-testing-2022-01-04-1159-distro-default-smithi/6595716/orig.config.yaml 2022-01-05T05:16:45.910 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-04_22:01:43-rbd-wip-yuri7-testing-2022-01-04-1159-distro-default-smithi/6595716 2022-01-05T05:16:45.911 INFO:teuthology.dispatcher.supervisor:Job PID: 23308 2022-01-05T05:16:45.911 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-05T17:17:40.471 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-05T17:17:40.665 INFO:teuthology.kill:Killing Pids: {23308} 2022-01-05T17:17:41.879 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2022-01-05T17:17:41.880 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2022-01-05T17:17:41.881 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['client.0'] 2022-01-05T17:17:41.881 INFO:teuthology.misc:Compressing logs... 2022-01-05T17:17:41.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2022-01-05T17:17:42.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2022-01-05T17:17:42.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2022-01-05T17:17:42.639 INFO:teuthology.misc:Compressing logs... 2022-01-05T17:17:43.556 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-01-05T17:17:44.090 INFO:teuthology.orchestra.run.smithi039.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-01-05T17:17:49.518 INFO:teuthology.misc:Compressing logs... 2022-01-05T17:18:20.716 INFO:teuthology.kill:No teuthology processes running 2022-01-05T17:18:21.833 INFO:teuthology.kill:Nuking machines: ['smithi039', 'smithi160', 'smithi175'] 2022-01-05T17:18:22.781 INFO:teuthology.kill:2022-01-05 17:18:22,780.780 INFO:teuthology.nuke:targets: 2022-01-05T17:18:22.782 INFO:teuthology.kill: smithi039.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMQuXYaTCF0CNoygfcKMMr1C4c+eqFB86Qt7YYYxMILvj8UUb6p2IEcr+JByVP4zRx+4L70M05ZnUGuSpH5LGCC2CL3QerfDDPS5WsV6kriQ08cmfEQgdSKQ9g5TsENJmxsZMNEKNJ/dp9/sugq6GYZ5AvslOkQLgJtBiwh9ddAVrLj+ucI1PTMPMAW55LM9Eu8p/cF++rTpOAEOYqZ2HtpCl49e5LNxHb47H6MQmExXlQucelWjtQua6ljwvtRq7oGpYu1auMlYpO6vICx2mlP0dn4pN4XfX6D7qvjQPfMzMd3Y1cypl/JcoyxBEZ1GBWEfJ9blegrt7G0PyKQCdTzQ/ofWA2gFbQu0AfA0mMwBZh8UrmNmZp/qy7UtkzpIpIog7ctKzDJ6/gNuY8BE7dwUDJBHsdnEcu9Dwk0PLOY6jWKwkG2VRkG5rnYdDmyo/Ke7Va751CS/ih5lu5yl6uTq6wwpYpL18NIDVxVJqHWrnre36+s3TBZ98KE6MnmP0= 2022-01-05T17:18:22.782 INFO:teuthology.kill: smithi160.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDVqpqrydGEfhs0hvUM2sYlpnmnc8/RoVi8IHOxnCJbNCubGyZdDoZfvcm78tWy/qDvUgyhpgUAZP2eg2iNFYSNHCVieGc/T9Xhf9SrhXNQd8km2vaAEMyZAfLiYr5Xb0Ndf5WXxWlO0w2oM/wStgnCWTMKVGxWwP60CpqCvs4UCOTebpKfA/kGshWjvWBNBHcTjCzWgz6HByuUtR1WCZPySPvWVhuUgDV0B4Q1ziDJhvrP3+3h11ZiPv50W3cnirn6Bg9vqMMIwsME/LcOEVmLgNFV4WfkpvmpM20G66OpbS6r46VU40YPj4qVCg0qliGzMo/cjl3j742YQCXvaQDB5bCRxZCkYERR/uLvinPKc7Yi9v+nWqMAZ/goIMHnA1h+kLYfsJpomAqDLRhqRzH/KP9/ipnky76OF3aVCUvwwcQfEIJpKXfUsHDMzRdSUqEyIEopIOiDr4qr4FltfofTHS+UzT1QPeWT07GZXJ09RV5LWNHv6nwN/7cwAv2o2pM= 2022-01-05T17:18:22.782 INFO:teuthology.kill: smithi175.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDK3y5jXGsCLUIH5vTMrku07iH77kEy/+TPQVriVs/uA4Sp5XMMU9YCiZbTEkD5ib8Jq+BdWOC4PRKPt95kLgoAtNpFKYfL1twIWowWknvewSr3Pe3/S6tff65LlFRUfSFhvhJDgrylhuiSNydO2/8+w6p3UdymZxn9yiyLL6udfhdIsG6cDeHRQ725tT7y5/pMz9w1C+h6TQM6oEnQzLJHJ/73Htd9916qPcFJSvUQWvvimwQcKiI1d3J+JoaIutT5XUXi4HTLgMTGm9dboinhYbzLy0C7np2gCOcRKERAEQCJGNE9UQf3ljwYY81/ZvSqTtS4qoOVprCCTt5Tvvn9ITKViGj1wRY1RKenBjsJ/6s44E7EdgGZL1J3e2++eL3g93VJrZ/RZ2Jj1ourD8N+UPNI8XZ7clnFaAPdnV/aMViRyGzzl71VIb7UcaorWGDCftzLDoOxQFERoE471lIASJQDuFEDYkoRZD5TcaP9jZEJptVs1MXgefVTPPA8Hks= 2022-01-05T17:18:22.809 INFO:teuthology.kill:2022-01-05 17:18:22,809.809 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-05T17:18:22.813 INFO:teuthology.kill:2022-01-05 17:18:22,813.813 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-05T17:18:22.815 INFO:teuthology.kill:2022-01-05 17:18:22,814.814 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-05T17:18:22.898 INFO:teuthology.kill:2022-01-05 17:18:22,897.897 INFO:teuthology.orchestra.console:Power off smithi039 2022-01-05T17:18:22.912 INFO:teuthology.kill:2022-01-05 17:18:22,911.911 INFO:teuthology.orchestra.console:Power off smithi175 2022-01-05T17:18:22.922 INFO:teuthology.kill:2022-01-05 17:18:22,922.922 INFO:teuthology.orchestra.console:Power off smithi160 2022-01-05T17:18:31.281 INFO:teuthology.kill:2022-01-05 17:18:31,281.281 INFO:teuthology.orchestra.console:Power off for smithi039 completed 2022-01-05T17:18:31.286 INFO:teuthology.kill:2022-01-05 17:18:31,286.286 INFO:teuthology.orchestra.console:Power off for smithi175 completed 2022-01-05T17:18:31.428 INFO:teuthology.kill:2022-01-05 17:18:31,428.428 INFO:teuthology.lock.ops:unlocked: smithi039.front.sepia.ceph.com 2022-01-05T17:18:31.432 INFO:teuthology.kill:2022-01-05 17:18:31,432.432 INFO:teuthology.lock.ops:unlocked: smithi175.front.sepia.ceph.com 2022-01-05T17:18:35.404 INFO:teuthology.kill:2022-01-05 17:18:35,403.403 INFO:teuthology.orchestra.console:Power off for smithi160 completed 2022-01-05T17:18:35.560 INFO:teuthology.kill:2022-01-05 17:18:35,560.560 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com 2022-01-05T17:20:35.852 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-05T17:20:35.992 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi039 but it was locked by another job: /home/teuthworker/archive/mchangir-2022-01-05_12:54:43-fs-wip-mchangir-mds-retry-backtrace-validation-distro-default-smithi/6596318 2022-01-05T17:20:35.992 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi160 but it was locked by another job: /home/teuthworker/archive/mchangir-2022-01-05_12:54:43-fs-wip-mchangir-mds-retry-backtrace-validation-distro-default-smithi/6596321 2022-01-05T17:20:35.993 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi175 but it was locked by another job: /home/teuthworker/archive/mchangir-2022-01-05_12:54:43-fs-wip-mchangir-mds-retry-backtrace-validation-distro-default-smithi/6596318