2021-12-07T00:26:05.111 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-07T00:26:05.262 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging 2021-12-07T00:26:05.263 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server 2021-12-07T00:26:05.263 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2021-12-07T00:26:05.283 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete 2021-12-07T00:26:05.283 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging 2021-12-07T00:26:05.284 INFO:teuthology.lock.ops:Updating [ubuntu@smithi174.front.sepia.ceph.com]: reset os type and version on server 2021-12-07T00:26:05.284 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2021-12-07T00:26:05.316 INFO:teuthology.lock.ops:Node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging is complete 2021-12-07T00:26:05.495 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 8.3 2021-12-07T00:26:05.538 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 8.3 2021-12-07T00:26:05.819 INFO:teuthology.orchestra.console:Power off smithi110 2021-12-07T00:26:05.871 INFO:teuthology.orchestra.console:Power off smithi174 2021-12-07T00:26:06.019 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2021-12-07T00:26:06.121 INFO:teuthology.orchestra.console:Power on smithi174 2021-12-07T00:26:14.204 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-12-07T00:26:14.306 INFO:teuthology.orchestra.console:Power on smithi110 2021-12-07T00:26:14.483 INFO:teuthology.orchestra.console:Power on for smithi174 completed 2021-12-07T00:26:14.585 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2021-12-07T00:26:22.681 INFO:teuthology.orchestra.console:Power on for smithi110 completed 2021-12-07T00:26:22.782 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish 2021-12-07T00:31:04.139 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-07T00:31:04.447 INFO:teuthology.provision.fog.smithi174:Node is ready 2021-12-07T00:31:04.464 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2021-12-07T00:31:04.521 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2021-12-07T00:31:04.847 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2021-12-07T00:31:12.195 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-07T00:31:12.495 INFO:teuthology.provision.fog.smithi110:Node is ready 2021-12-07T00:31:12.551 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com 2021-12-07T00:31:12.606 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110 2021-12-07T00:31:12.935 INFO:teuthology.provision.fog.smithi110:Deploy complete! 2021-12-07T00:31:13.133 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com 2021-12-07T00:31:13.134 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-07T00:31:13.156 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2021-12-07T00:31:13.156 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-07T00:31:13.175 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server 2021-12-07T00:31:13.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2021-12-07T00:31:13.372 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2021-12-07T00:31:13.426 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Linux" 2021-12-07T00:31:13.426 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8" 2021-12-07T00:31:13.426 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2021-12-07T00:31:13.427 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2021-12-07T00:31:13.427 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8" 2021-12-07T00:31:13.427 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8" 2021-12-07T00:31:13.427 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-07T00:31:13.428 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2021-12-07T00:31:13.428 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-07T00:31:13.428 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2021-12-07T00:31:13.428 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-07T00:31:13.429 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-07T00:31:13.429 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-07T00:31:13.430 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2021-12-07T00:31:13.449 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2021-12-07T00:31:13.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2021-12-07T00:31:13.720 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2021-12-07T00:31:13.783 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Linux" 2021-12-07T00:31:13.784 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="8" 2021-12-07T00:31:13.784 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2021-12-07T00:31:13.784 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2021-12-07T00:31:13.784 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="8" 2021-12-07T00:31:13.785 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el8" 2021-12-07T00:31:13.785 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-07T00:31:13.785 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2021-12-07T00:31:13.785 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-07T00:31:13.786 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2021-12-07T00:31:13.786 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-07T00:31:13.786 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-07T00:31:13.786 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-07T00:31:13.787 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2021-12-07T00:31:13.885 INFO:teuthology.dispatcher.supervisor:Running job 6547944 2021-12-07T00:31:13.886 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547944 --name benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547944/orig.config.yaml 2021-12-07T00:31:13.889 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547944 2021-12-07T00:31:13.890 INFO:teuthology.dispatcher.supervisor:Job PID: 4702 2021-12-07T00:31:13.890 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-07T12:32:08.091 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-07T12:32:08.326 INFO:teuthology.kill:Killing Pids: {4702} 2021-12-07T12:32:09.523 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-12-07T12:32:09.524 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-12-07T12:32:09.524 INFO:teuthology.misc:Compressing logs... 2021-12-07T12:32:09.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2021-12-07T12:32:09.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2021-12-07T12:32:10.101 INFO:teuthology.misc:Compressing logs... 2021-12-07T12:32:10.918 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-07T12:32:11.128 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-07T12:32:16.184 INFO:teuthology.misc:Compressing logs... 2021-12-07T12:32:37.306 INFO:teuthology.kill:No teuthology processes running 2021-12-07T12:32:38.454 INFO:teuthology.kill:Nuking machines: ['smithi110', 'smithi174'] 2021-12-07T12:32:39.397 INFO:teuthology.kill:2021-12-07 12:32:39,397.397 INFO:teuthology.nuke:targets: 2021-12-07T12:32:39.398 INFO:teuthology.kill: smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCx+15aBvppPHI7W8v7kmQGM5HC1ONLU8Dk+T8ERtLdFWFdGuOUdQUKf4og/T42T4ttUUp87Ptq3I/A+9sno1OmYSSteAt9EPm7b4n6Uzn01iVk/etdfmzy66aJddVnyT6HLwmsGNO3Te5jCW6Tgs82d/t6hJoUPzQ91Bm5x+3pWGY0e1GuM9IlN+l4GwgHfLeOwHGMl3VPJXqg9LcHjyyFiFaMP+lt3uY3+6IlMN+nYf21vkpmvZvGBo7iZCAS+9h7IeGOyvJ/dC3CVrbf1BOzhJT+lFCE2jKnK/rDYQCMLnUjDNCGk1WPs/fJtjFRoUEbEXrsxX5OfyBcCWZH7/FkrfdSs57mJKy9wYXjFT57EQZhByaiqd8YZ0Uzy0LQoRPyL5I560SBsoafD+660lzu4Thu+ypFykbCRjJcM5z3i9KuGKEY46q36RPaNT2MBvLee9zkDN6ImQgFitesS6+b3WbEJ8Z6K6eQ/Qvxj2Ow7jMA/igzoGQx+rZG2P+zan8= 2021-12-07T12:32:39.399 INFO:teuthology.kill: smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfVaJeBybeUKjY5iKP/aP8Yrd5I+qnPqOkT5inWGfI/UMtvHXLp0v0ngM388ohBsh+rLsCc16t5g1dMR/YThvEQmqZbGCTxXG6EH51NAKRMMnXNBrDbxOKty7RyR+1fO6w0XXL4Szb8MzTSLZvMeNBvn3wqKxl4O74JuG6RiRSwOXNtfsm7VFVhJZzBUJtPrhYclL0YETsscDMsK7LUN8/iRqaMDC6572u1vBmM3NreQAeG1S8eRKI/hDL65NaCvObg1SpGk4obL++Q8xGpD4wNMzgeom3fMRkR7gctxZG/iQto/z5IAPVGBHG0n9zmLFFeNj44l1jqrcVZfmFWh25w0BOg6aWG+QJNc9Z/9B0q7VztOtIjTJ2xpSsyo9Aihlu7ivvEm7LzhWRRl1PyMpUacMmto1v33W5zw3sM3mZSYr85+lDaBhhLw1Mlkn8Sr5Ej90gzmWb6dzB0agzE7etGPyo7X2V1URZ5tuhzEnj2Al2APpslFaaExM8AJYfVeU= 2021-12-07T12:32:39.424 INFO:teuthology.kill:2021-12-07 12:32:39,424.424 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-07T12:32:39.427 INFO:teuthology.kill:2021-12-07 12:32:39,427.427 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-07T12:32:39.503 INFO:teuthology.kill:2021-12-07 12:32:39,503.503 INFO:teuthology.orchestra.console:Power off smithi110 2021-12-07T12:32:39.517 INFO:teuthology.kill:2021-12-07 12:32:39,517.517 INFO:teuthology.orchestra.console:Power off smithi174 2021-12-07T12:32:47.876 INFO:teuthology.kill:2021-12-07 12:32:47,876.876 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-12-07T12:32:48.024 INFO:teuthology.kill:2021-12-07 12:32:48,023.023 INFO:teuthology.lock.ops:unlocked: smithi110.front.sepia.ceph.com 2021-12-07T12:32:51.992 INFO:teuthology.kill:2021-12-07 12:32:51,992.992 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2021-12-07T12:32:52.139 INFO:teuthology.kill:2021-12-07 12:32:52,139.139 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com 2021-12-07T12:34:52.466 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-07T12:34:52.579 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi110 but it was locked by another job: /home/teuthworker/archive/ideepika-2021-12-07_12:24:56-rbd:persistent-writeback-cache-master-distro-default-smithi/6550256 2021-12-07T12:34:52.579 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi174 but it was locked by another job: /home/teuthworker/archive/ideepika-2021-12-07_12:24:56-rbd:persistent-writeback-cache-master-distro-default-smithi/6550256