2022-03-14T13:36:09.172 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-14T13:36:09.291 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging 2022-03-14T13:36:09.293 INFO:teuthology.lock.ops:Updating [ubuntu@smithi017.front.sepia.ceph.com]: reset os type and version on server 2022-03-14T13:36:09.293 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2022-03-14T13:36:09.311 INFO:teuthology.lock.ops:Node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging is complete 2022-03-14T13:36:09.312 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging 2022-03-14T13:36:09.312 INFO:teuthology.lock.ops:Updating [ubuntu@smithi029.front.sepia.ceph.com]: reset os type and version on server 2022-03-14T13:36:09.312 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2022-03-14T13:36:09.330 INFO:teuthology.lock.ops:Node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging is complete 2022-03-14T13:36:09.527 INFO:teuthology.provision.fog.smithi017:Scheduling deploy of centos 8.stream 2022-03-14T13:36:09.549 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of centos 8.stream 2022-03-14T13:36:09.830 INFO:teuthology.orchestra.console:Power off smithi017 2022-03-14T13:36:09.845 INFO:teuthology.orchestra.console:Power off smithi029 2022-03-14T13:36:18.203 INFO:teuthology.orchestra.console:Power off for smithi017 completed 2022-03-14T13:36:18.209 INFO:teuthology.orchestra.console:Power off for smithi029 completed 2022-03-14T13:36:18.305 INFO:teuthology.orchestra.console:Power on smithi017 2022-03-14T13:36:18.313 INFO:teuthology.orchestra.console:Power on smithi029 2022-03-14T13:36:26.671 INFO:teuthology.orchestra.console:Power on for smithi017 completed 2022-03-14T13:36:26.678 INFO:teuthology.orchestra.console:Power on for smithi029 completed 2022-03-14T13:36:26.773 INFO:teuthology.provision.fog.smithi017:Waiting for deploy to finish 2022-03-14T13:36:26.779 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish 2022-03-14T13:40:09.072 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-14T13:40:28.799 INFO:teuthology.provision.fog.smithi017:Node is ready 2022-03-14T13:40:28.822 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017.front.sepia.ceph.com 2022-03-14T13:40:28.883 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17 smithi017.front.sepia.ceph.com smithi017 2022-03-14T13:40:29.271 INFO:teuthology.provision.fog.smithi017:Deploy complete! 2022-03-14T13:41:13.608 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-14T13:41:14.032 INFO:teuthology.provision.fog.smithi029:Node is ready 2022-03-14T13:41:14.052 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com 2022-03-14T13:41:14.111 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029 2022-03-14T13:41:14.702 INFO:teuthology.provision.fog.smithi029:Deploy complete! 2022-03-14T13:41:14.854 INFO:teuthology.lock.ops:Checking smithi017.front.sepia.ceph.com 2022-03-14T13:41:14.855 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-14T13:41:14.875 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com 2022-03-14T13:41:14.876 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-14T13:41:14.894 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: set os type and version on server 2022-03-14T13:41:14.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2022-03-14T13:41:15.151 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2022-03-14T13:41:15.210 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream" 2022-03-14T13:41:15.210 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8" 2022-03-14T13:41:15.210 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos" 2022-03-14T13:41:15.210 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora" 2022-03-14T13:41:15.211 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8" 2022-03-14T13:41:15.211 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8" 2022-03-14T13:41:15.211 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-14T13:41:15.211 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2022-03-14T13:41:15.212 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-14T13:41:15.212 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/" 2022-03-14T13:41:15.212 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-14T13:41:15.212 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-14T13:41:15.213 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-14T13:41:15.214 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2022-03-14T13:41:15.232 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server 2022-03-14T13:41:15.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2022-03-14T13:41:15.428 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2022-03-14T13:41:15.486 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream" 2022-03-14T13:41:15.487 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8" 2022-03-14T13:41:15.487 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos" 2022-03-14T13:41:15.487 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora" 2022-03-14T13:41:15.487 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8" 2022-03-14T13:41:15.488 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8" 2022-03-14T13:41:15.488 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-14T13:41:15.488 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2022-03-14T13:41:15.488 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-14T13:41:15.489 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/" 2022-03-14T13:41:15.489 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-14T13:41:15.489 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-14T13:41:15.489 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-14T13:41:15.490 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2022-03-14T13:41:15.588 INFO:teuthology.dispatcher.supervisor:Running job 6735871 2022-03-14T13:41:15.589 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735871 --name vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} -- /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735871/orig.config.yaml 2022-03-14T13:41:15.592 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735871 2022-03-14T13:41:15.593 INFO:teuthology.dispatcher.supervisor:Job PID: 21252 2022-03-14T13:41:15.593 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-14T20:11:44.157 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-14T20:11:44.354 INFO:teuthology.kill:Killing Pids: {21252} 2022-03-14T20:11:45.560 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-14T20:11:45.561 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-14T20:11:45.562 INFO:teuthology.misc:Compressing logs... 2022-03-14T20:11:45.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2022-03-14T20:11:45.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2022-03-14T20:11:46.149 INFO:teuthology.misc:Compressing logs... 2022-03-14T20:11:46.394 INFO:teuthology.misc:Compressing logs... 2022-03-14T20:13:52.644 INFO:teuthology.kill:No teuthology processes running 2022-03-14T20:13:53.778 INFO:teuthology.kill:Nuking machines: ['smithi017', 'smithi029'] 2022-03-14T20:13:54.743 INFO:teuthology.kill:2022-03-14 20:13:54,743.743 INFO:teuthology.nuke:targets: 2022-03-14T20:13:54.744 INFO:teuthology.kill: smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4Ey/4Prdw1vcs3RBigV/tKfDHKwO0XnqBWcZSLReAWt6+f6Ankh3piSuKk3VE4CeQWVIxj7QotdtEJ3jqh5LyPsCObdQVBjIc5lr87SYiOU6u0DIS5lkh7YTWB7GFsT7igl9csvnhyvQ5TitDkbfHFZgDLsOsQmGZh42+/EFdD/bAraDHJMG+bfmJLHu9hHA7VIQkgc+MymqNqoAGjbatA9ar4oMkyVlxYh/f1phxl8F4krNJ0CBTQz4c6F44r4EBZbOZG12JvV/6Gc3S7EWTQHV/4H+/gRqn8IXs0K0dcQw8NB9msjm1ANO+SOYI9VG6kMqTUn419dlVY/1+DbWo5o961FP8fkBC//9CTpCi5hkISMyxkcg4uSFbcxXdQkKqoujFif4CuqJESsTz3KYRW0/Pz89OkgD/K33vMzJi1UvRbRvGiHEArE6QirgCTPDuPq/pqys4vjCbNhlNBuqqQlFhICvIuZCILZcSmu/7nA9yP9OxRnfknH0a0mJkdPM= 2022-03-14T20:13:54.745 INFO:teuthology.kill: smithi029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRNTzYKv0YVY8iGmuM2y5Tdz6/3TC9MwWr0uyyRFY6S2BVywRHHMjCKoOiy52UkZyfDQXNpXwLQWfLqQtwbrukamT63guRZIJq7QbbbGc7426ItiPNeAYgloawrFl1Y7lNVJ06Vit8xKBvDQBPE42+m8RZ9WiZou6A0OGlHuzgJ9/PjYqbcNTmBGe+194DMak/zCo6swPmFdhfNL7UvlYU7fySGtAt5aExv5GN74BrzHvHzYMM/t8sA9p4BBo68zy5gnpzqvqEzYoAD1vBE0TQVhGr4+5OYYhjyB+jyRPjMl1Bc9XGAv1/bU4ohI8ssRkgHKkLXH+6rWNeKQX15K1zhwe2yx0SzqJJqYG8bWfKA9kQJHVccmUU8n6iOuthfLjBDQ8a5zJNNKDxxEq6gcR91bH8KyElXQ4XZ0qGh8cwJmrPwKjuFYPxkQ4qtig5Yzw+vtSiVYp2sKii6UdqZQ0kXlQf54oV5dyW+4/NK+HZv9OSnmlBCP8dMHhXSSiPvXM= 2022-03-14T20:13:54.770 INFO:teuthology.kill:2022-03-14 20:13:54,770.770 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-14T20:13:54.773 INFO:teuthology.kill:2022-03-14 20:13:54,773.773 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-14T20:13:54.849 INFO:teuthology.kill:2022-03-14 20:13:54,849.849 INFO:teuthology.orchestra.console:Power off smithi029 2022-03-14T20:13:54.865 INFO:teuthology.kill:2022-03-14 20:13:54,864.864 INFO:teuthology.orchestra.console:Power off smithi017 2022-03-14T20:14:03.219 INFO:teuthology.kill:2022-03-14 20:14:03,219.219 INFO:teuthology.orchestra.console:Power off for smithi029 completed 2022-03-14T20:14:03.226 INFO:teuthology.kill:2022-03-14 20:14:03,225.225 INFO:teuthology.orchestra.console:Power off for smithi017 completed 2022-03-14T20:14:03.373 INFO:teuthology.kill:2022-03-14 20:14:03,372.372 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com 2022-03-14T20:14:03.376 INFO:teuthology.kill:2022-03-14 20:14:03,376.376 INFO:teuthology.lock.ops:unlocked: smithi017.front.sepia.ceph.com 2022-03-14T20:16:03.711 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-14T20:16:03.837 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi017 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-14_18:47:44-rados-wip-yuri3-testing-2022-03-14-0946-distro-default-smithi/6736267 2022-03-14T20:16:03.837 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi029 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-14_18:47:44-rados-wip-yuri3-testing-2022-03-14-0946-distro-default-smithi/6736267