2022-03-14T15:58:45.396 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-14T15:58:45.519 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging 2022-03-14T15:58:45.520 INFO:teuthology.lock.ops:Updating [ubuntu@smithi134.front.sepia.ceph.com]: reset os type and version on server 2022-03-14T15:58:45.521 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2022-03-14T15:58:45.539 INFO:teuthology.lock.ops:Node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging is complete 2022-03-14T15:58:45.539 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging 2022-03-14T15:58:45.540 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server 2022-03-14T15:58:45.540 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-03-14T15:58:45.559 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete 2022-03-14T15:58:45.748 INFO:teuthology.provision.fog.smithi134:Scheduling deploy of centos 8.stream 2022-03-14T15:58:45.758 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of centos 8.stream 2022-03-14T15:58:46.057 INFO:teuthology.orchestra.console:Power off smithi134 2022-03-14T15:58:46.072 INFO:teuthology.orchestra.console:Power off smithi185 2022-03-14T15:58:46.200 INFO:teuthology.orchestra.console:Power off for smithi134 completed 2022-03-14T15:58:46.211 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-03-14T15:58:46.302 INFO:teuthology.orchestra.console:Power on smithi134 2022-03-14T15:58:46.312 INFO:teuthology.orchestra.console:Power on smithi185 2022-03-14T15:58:50.562 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2022-03-14T15:58:50.663 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2022-03-14T15:58:54.731 INFO:teuthology.orchestra.console:Power on for smithi134 completed 2022-03-14T15:58:54.832 INFO:teuthology.provision.fog.smithi134:Waiting for deploy to finish 2022-03-14T16:03:26.385 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-14T16:03:26.662 INFO:teuthology.provision.fog.smithi134:Node is ready 2022-03-14T16:03:26.718 INFO:teuthology.orchestra.run.smithi134.stdout:smithi134.front.sepia.ceph.com 2022-03-14T16:03:26.776 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.15.134 smithi134.front.sepia.ceph.com smithi134 2022-03-14T16:03:27.118 INFO:teuthology.provision.fog.smithi134:Deploy complete! 2022-03-14T16:03:37.384 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-14T16:03:37.671 INFO:teuthology.provision.fog.smithi185:Node is ready 2022-03-14T16:03:37.689 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2022-03-14T16:03:37.747 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2022-03-14T16:03:38.088 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2022-03-14T16:03:38.277 INFO:teuthology.lock.ops:Checking smithi134.front.sepia.ceph.com 2022-03-14T16:03:38.278 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-14T16:03:38.297 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2022-03-14T16:03:38.298 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-14T16:03:38.316 INFO:teuthology.lock.ops:Updating [smithi134.front.sepia.ceph.com]: set os type and version on server 2022-03-14T16:03:38.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2022-03-14T16:03:38.557 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2022-03-14T16:03:38.614 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2022-03-14T16:03:38.614 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8" 2022-03-14T16:03:38.615 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2022-03-14T16:03:38.615 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2022-03-14T16:03:38.615 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8" 2022-03-14T16:03:38.616 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8" 2022-03-14T16:03:38.616 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-14T16:03:38.616 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2022-03-14T16:03:38.616 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-14T16:03:38.616 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2022-03-14T16:03:38.617 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-14T16:03:38.617 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-14T16:03:38.617 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-14T16:03:38.618 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2022-03-14T16:03:38.636 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2022-03-14T16:03:38.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-03-14T16:03:38.824 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2022-03-14T16:03:38.880 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="CentOS Stream" 2022-03-14T16:03:38.881 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8" 2022-03-14T16:03:38.881 INFO:teuthology.orchestra.run.smithi185.stdout:ID="centos" 2022-03-14T16:03:38.881 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="rhel fedora" 2022-03-14T16:03:38.882 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8" 2022-03-14T16:03:38.882 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8" 2022-03-14T16:03:38.882 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-14T16:03:38.882 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31" 2022-03-14T16:03:38.882 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-14T16:03:38.883 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://centos.org/" 2022-03-14T16:03:38.883 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-14T16:03:38.883 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-14T16:03:38.884 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-14T16:03:38.884 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-03-14T16:03:38.986 INFO:teuthology.dispatcher.supervisor:Running job 6735916 2022-03-14T16:03:38.986 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735916 --name vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} -- /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735916/orig.config.yaml 2022-03-14T16:03:38.990 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735916 2022-03-14T16:03:38.991 INFO:teuthology.dispatcher.supervisor:Job PID: 22526 2022-03-14T16:03:38.991 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-14T22:34:07.774 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-14T22:34:07.960 INFO:teuthology.kill:Killing Pids: {22526} 2022-03-14T22:34:09.180 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-14T22:34:09.181 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-14T22:34:09.182 INFO:teuthology.misc:Compressing logs... 2022-03-14T22:34:09.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2022-03-14T22:34:09.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-03-14T22:34:09.784 INFO:teuthology.misc:Compressing logs... 2022-03-14T22:34:10.016 INFO:teuthology.misc:Compressing logs... 2022-03-14T22:36:35.599 INFO:teuthology.kill:No teuthology processes running 2022-03-14T22:36:36.786 INFO:teuthology.kill:Nuking machines: ['smithi134', 'smithi185'] 2022-03-14T22:36:37.831 INFO:teuthology.kill:2022-03-14 22:36:37,831.831 INFO:teuthology.nuke:targets: 2022-03-14T22:36:37.832 INFO:teuthology.kill: smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHAsoccbWwKE3INNc3YkhsQ8zRB0gop/scC3E1RqL14EQqaC1TVlfmA1EKl5yYtYhjHmx6wL0uqtDhY8rYXkSAVrYnO/BwPFxA8x0wCriPRgpUlK+K0nULj9nyYhOp8aYChM+litsugL2GF0/CHId0ogC2xUyY1eMBYqNpN3AHuj81cMxrT+a7Av3WsUiSTZu2Wrl3FqKrNoVlh4RgHulCCATj7H6zURDB1eaAl0MHs01RBUIk6oxDLfjIvXelc8jRyy3Q+AalRei0ZVzH/las+TvEWCaYEaoFa0Rk4RVPMsbcWpsbMNndmXh/Lk2JIXH9hfdzESo9W7zqbUDVVU6r6TZfizmzL2Y2bAXx5RPrxCq1i3ZaG+xG+nIzWg8N+SqxJCxZyysPI+A7BuaHRcKfO1pyfZRvmoe2kFRDxEWar8DyRHrtmwmzfwQ5sH1cJbKyQ+Yu8VYdquRd58LVaheo7/rvwtBfFGTDPQk3hnT4mkyCE/+OgdETm+T9TXKc+fM= 2022-03-14T22:36:37.832 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDF5sp+Y9TaZ79vVjdinT+v5axK0DllRQO76sLeTy1SiQmu1QGF5sJzPMF0fvKNOZWyJHY55o0DOh0aIv7x2K6WpumiapgvZwI6SXhkrhHx7nED5lS++4t527amMZYHjAxpiJ/vwMwyu7azodNH42GKccjIEoGqr85ROkVKRPOsrwFBkqcFMA+vNFRQFExbEyzT85clI16zTLc+MZuyVzTMga7GwchoeTyZDeWFpXPvf/sHbtnZ9HKbUpQ0W85bscsI7GDNHoB3Ew6ggy7/AcXF8B+uVsUKixEEcsowob6UursrsdOgVg+hkR6B/BCR6UrXlY19T7D1Du8BWlSAO7fYZKX+P24thsHSCFU6C/EK72iRVSveIVmYMZ6iXxHKxyllUV4/Xv8GEkM5uQKFIUizMrdlMUSE/K1JJKRAVlnISh/J3uakGKpgrrjb+vdxW4zV+Ye3qs3zAreLwAbTDYutCdgZzEaPE+p+M8LCY3pwV4xnxWphEeXqwSQC+gJV2UM= 2022-03-14T22:36:37.859 INFO:teuthology.kill:2022-03-14 22:36:37,858.858 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-14T22:36:37.862 INFO:teuthology.kill:2022-03-14 22:36:37,862.862 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-14T22:36:37.936 INFO:teuthology.kill:2022-03-14 22:36:37,936.936 INFO:teuthology.orchestra.console:Power off smithi134 2022-03-14T22:36:37.950 INFO:teuthology.kill:2022-03-14 22:36:37,950.950 INFO:teuthology.orchestra.console:Power off smithi185 2022-03-14T22:36:50.418 INFO:teuthology.kill:2022-03-14 22:36:50,418.418 INFO:teuthology.orchestra.console:Power off for smithi134 completed 2022-03-14T22:36:50.424 INFO:teuthology.kill:2022-03-14 22:36:50,424.424 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-03-14T22:36:50.565 INFO:teuthology.kill:2022-03-14 22:36:50,564.564 INFO:teuthology.lock.ops:unlocked: smithi134.front.sepia.ceph.com 2022-03-14T22:36:50.588 INFO:teuthology.kill:2022-03-14 22:36:50,588.588 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com 2022-03-14T22:38:50.913 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-14T22:38:51.028 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi134 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-14_18:47:44-rados-wip-yuri3-testing-2022-03-14-0946-distro-default-smithi/6736683 2022-03-14T22:38:51.028 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-14_18:47:44-rados-wip-yuri3-testing-2022-03-14-0946-distro-default-smithi/6736683