2022-03-20T07:19:14.368 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-20T07:19:14.486 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi005.front.sepia.ceph.com' reimaging 2022-03-20T07:19:14.487 INFO:teuthology.lock.ops:Updating [ubuntu@smithi005.front.sepia.ceph.com]: reset os type and version on server 2022-03-20T07:19:14.487 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2022-03-20T07:19:14.509 INFO:teuthology.lock.ops:Node 'ubuntu@smithi005.front.sepia.ceph.com' reimaging is complete 2022-03-20T07:19:14.510 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi037.front.sepia.ceph.com' reimaging 2022-03-20T07:19:14.510 INFO:teuthology.lock.ops:Updating [ubuntu@smithi037.front.sepia.ceph.com]: reset os type and version on server 2022-03-20T07:19:14.511 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2022-03-20T07:19:14.531 INFO:teuthology.lock.ops:Node 'ubuntu@smithi037.front.sepia.ceph.com' reimaging is complete 2022-03-20T07:19:14.726 INFO:teuthology.provision.fog.smithi005:Scheduling deploy of centos 8.stream 2022-03-20T07:19:14.733 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of centos 8.stream 2022-03-20T07:19:15.093 INFO:teuthology.orchestra.console:Power off smithi037 2022-03-20T07:19:15.106 INFO:teuthology.orchestra.console:Power off smithi005 2022-03-20T07:19:23.465 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2022-03-20T07:19:23.472 INFO:teuthology.orchestra.console:Power off for smithi005 completed 2022-03-20T07:19:23.567 INFO:teuthology.orchestra.console:Power on smithi037 2022-03-20T07:19:23.575 INFO:teuthology.orchestra.console:Power on smithi005 2022-03-20T07:19:31.940 INFO:teuthology.orchestra.console:Power on for smithi037 completed 2022-03-20T07:19:32.042 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish 2022-03-20T07:19:32.063 INFO:teuthology.orchestra.console:Power on for smithi005 completed 2022-03-20T07:19:32.164 INFO:teuthology.provision.fog.smithi005:Waiting for deploy to finish 2022-03-20T07:23:14.362 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-20T07:23:14.393 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-20T07:23:34.905 INFO:teuthology.provision.fog.smithi005:Node is ready 2022-03-20T07:23:34.928 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005.front.sepia.ceph.com 2022-03-20T07:23:34.988 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5 smithi005.front.sepia.ceph.com smithi005 2022-03-20T07:23:35.258 INFO:teuthology.provision.fog.smithi037:Node is ready 2022-03-20T07:23:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com 2022-03-20T07:23:35.347 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037 2022-03-20T07:23:35.366 INFO:teuthology.provision.fog.smithi005:Deploy complete! 2022-03-20T07:23:35.726 INFO:teuthology.provision.fog.smithi037:Deploy complete! 2022-03-20T07:23:35.878 INFO:teuthology.lock.ops:Checking smithi005.front.sepia.ceph.com 2022-03-20T07:23:35.879 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-20T07:23:35.899 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com 2022-03-20T07:23:35.899 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-20T07:23:35.919 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: set os type and version on server 2022-03-20T07:23:35.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2022-03-20T07:23:36.116 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2022-03-20T07:23:36.174 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2022-03-20T07:23:36.175 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8" 2022-03-20T07:23:36.175 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2022-03-20T07:23:36.175 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2022-03-20T07:23:36.175 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8" 2022-03-20T07:23:36.176 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8" 2022-03-20T07:23:36.176 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-20T07:23:36.176 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2022-03-20T07:23:36.176 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-20T07:23:36.176 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2022-03-20T07:23:36.177 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-20T07:23:36.177 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-20T07:23:36.177 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-20T07:23:36.178 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2022-03-20T07:23:36.198 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server 2022-03-20T07:23:36.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2022-03-20T07:23:36.396 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2022-03-20T07:23:36.455 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2022-03-20T07:23:36.455 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8" 2022-03-20T07:23:36.455 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2022-03-20T07:23:36.456 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2022-03-20T07:23:36.456 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8" 2022-03-20T07:23:36.456 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8" 2022-03-20T07:23:36.456 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-20T07:23:36.457 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2022-03-20T07:23:36.457 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-20T07:23:36.457 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2022-03-20T07:23:36.457 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-20T07:23:36.457 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-20T07:23:36.458 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-20T07:23:36.458 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2022-03-20T07:23:36.558 INFO:teuthology.dispatcher.supervisor:Running job 6747651 2022-03-20T07:23:36.558 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747651 --name vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} -- /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747651/orig.config.yaml 2022-03-20T07:23:36.562 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747651 2022-03-20T07:23:36.563 INFO:teuthology.dispatcher.supervisor:Job PID: 15628 2022-03-20T07:23:36.563 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-20T13:54:04.463 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-20T13:54:04.630 INFO:teuthology.kill:Killing Pids: {15628} 2022-03-20T13:54:05.891 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-20T13:54:05.892 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-20T13:54:05.893 INFO:teuthology.misc:Compressing logs... 2022-03-20T13:54:05.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2022-03-20T13:54:10.503 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2022-03-20T13:54:10.889 INFO:teuthology.misc:Compressing logs... 2022-03-20T13:54:11.122 INFO:teuthology.misc:Compressing logs... 2022-03-20T14:04:19.766 INFO:teuthology.kill:No teuthology processes running 2022-03-20T14:04:20.957 INFO:teuthology.kill:Nuking machines: ['smithi005', 'smithi037'] 2022-03-20T14:04:21.903 INFO:teuthology.kill:2022-03-20 14:04:21,903.903 INFO:teuthology.nuke:targets: 2022-03-20T14:04:21.904 INFO:teuthology.kill: smithi005.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW5noqGhC7tweAxxmV4F/j3RvBxax3dBPkGZU5pGDMW41o1VN996m3T1Tae0HiWflj3opl7Mvnw45xaQ+Ft8Rk1CfqxhzRTO6GowFiByNC3TiDjV+kStUc4NjkzswCtiCaB7gYosdr7fyVCzbrq31uOCFDxMA/otwghOWFft/imEnL0u3co8uE41QI/Mc0YG+kGBELr71BcxvitrNKZ6SzKiioKU0ZMoAZ15f00lt55TiatBvfvCW1Gbsz9n74898c+YrFFZo5p/vFF5x/Ipm/tutNB0KwAodkLHNoLQ3ISaV1KGuZ+Uo7rv5MXNrbBedOcC4pGueCsxNv4pMEEiChcJZd3kFfgQMu2MBbzBbzdu82xZ9t/i5272uIfThhrI3Pb2AHo9fLfM22QBANWYuN18l+tCwoCU4GjYOyo4m8dpLH+dHu9MpBeuZUKOa1L9BmaFA9zTdmVqgpcm+eti+XtNUxx6pt+du2h9L+C2/ijJVW13IHlMm4DG7gHglD+ws= 2022-03-20T14:04:21.904 INFO:teuthology.kill: smithi037.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtpEasaXRDLKC4ZJNb3jyCC+ur6pVuvzYVxDpDdGomKVnd/Tazaha1Y31QoqobvyVWnTnYc4WG47am6YZnKipEu78On65LLuPdzw2T98fHwAtP2yEpN6PRTt7MjoGEJ9+q56XZIXhwscVe4n1wZqXyffhiBiIaWrHm3nCfFp0nJvYXCruqPBox547F9ZJ/BPXCnyxQs/cVeDNbjsKS25Yfo9L0Mmnu/zld/tft1U5gxpY8IOk216YhL1Kto4XlOVAslrXh0npNR3CItMKOnj7Ky5kJlEzgEUb4XdwD956HgfLfy2KICeOYwqtvdp04ri2KHFKPH+oo1ELKpl9Xgrg1ZG/BL1jZ5tdsUb4nnp6d9t3a2AGWz4Hci9caXc1cXIkZZsb8ygIRaORRoiQQQV8LvIY1pktDv8tEIQK5UyEqHK2dERjCXPJMIDwdfNRAg4mVzSI9RQOv8orMLKlpOUrhCmIbBVt53tdrCTV7ODk6rRZwagyc8WF6GM3N6KiRQEE= 2022-03-20T14:04:21.929 INFO:teuthology.kill:2022-03-20 14:04:21,929.929 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-20T14:04:21.932 INFO:teuthology.kill:2022-03-20 14:04:21,932.932 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-20T14:04:22.009 INFO:teuthology.kill:2022-03-20 14:04:22,008.008 INFO:teuthology.orchestra.console:Power off smithi037 2022-03-20T14:04:22.021 INFO:teuthology.kill:2022-03-20 14:04:22,021.021 INFO:teuthology.orchestra.console:Power off smithi005 2022-03-20T14:04:30.383 INFO:teuthology.kill:2022-03-20 14:04:30,383.383 INFO:teuthology.orchestra.console:Power off for smithi005 completed 2022-03-20T14:04:30.530 INFO:teuthology.kill:2022-03-20 14:04:30,529.529 INFO:teuthology.lock.ops:unlocked: smithi005.front.sepia.ceph.com 2022-03-20T14:04:34.488 INFO:teuthology.kill:2022-03-20 14:04:34,487.487 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2022-03-20T14:04:34.631 INFO:teuthology.kill:2022-03-20 14:04:34,631.631 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com 2022-03-20T14:06:34.917 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-20T14:06:35.025 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi005 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747824 2022-03-20T14:06:35.025 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi037 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747824