2022-03-15T04:42:22.591 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-15T04:42:22.724 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi141.front.sepia.ceph.com' reimaging 2022-03-15T04:42:22.726 INFO:teuthology.lock.ops:Updating [ubuntu@smithi141.front.sepia.ceph.com]: reset os type and version on server 2022-03-15T04:42:22.727 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2022-03-15T04:42:22.745 INFO:teuthology.lock.ops:Node 'ubuntu@smithi141.front.sepia.ceph.com' reimaging is complete 2022-03-15T04:42:22.745 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi145.front.sepia.ceph.com' reimaging 2022-03-15T04:42:22.746 INFO:teuthology.lock.ops:Updating [ubuntu@smithi145.front.sepia.ceph.com]: reset os type and version on server 2022-03-15T04:42:22.746 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-03-15T04:42:22.766 INFO:teuthology.lock.ops:Node 'ubuntu@smithi145.front.sepia.ceph.com' reimaging is complete 2022-03-15T04:42:22.958 INFO:teuthology.provision.fog.smithi141:Scheduling deploy of centos 8.stream 2022-03-15T04:42:23.005 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of centos 8.stream 2022-03-15T04:42:23.348 INFO:teuthology.orchestra.console:Power off smithi141 2022-03-15T04:42:23.361 INFO:teuthology.orchestra.console:Power off smithi145 2022-03-15T04:42:31.718 INFO:teuthology.orchestra.console:Power off for smithi141 completed 2022-03-15T04:42:31.726 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-03-15T04:42:31.820 INFO:teuthology.orchestra.console:Power on smithi141 2022-03-15T04:42:31.829 INFO:teuthology.orchestra.console:Power on smithi145 2022-03-15T04:42:40.191 INFO:teuthology.orchestra.console:Power on for smithi141 completed 2022-03-15T04:42:40.199 INFO:teuthology.orchestra.console:Power on for smithi145 completed 2022-03-15T04:42:40.293 INFO:teuthology.provision.fog.smithi141:Waiting for deploy to finish 2022-03-15T04:42:40.300 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2022-03-15T04:46:46.984 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-15T04:46:53.194 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-15T04:47:12.441 INFO:teuthology.provision.fog.smithi145:Node is ready 2022-03-15T04:47:12.464 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2022-03-15T04:47:12.523 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2022-03-15T04:47:12.896 INFO:teuthology.provision.fog.smithi145:Deploy complete! 2022-03-15T04:47:18.194 INFO:teuthology.provision.fog.smithi141:Node is ready 2022-03-15T04:47:18.219 INFO:teuthology.orchestra.run.smithi141.stdout:smithi141.front.sepia.ceph.com 2022-03-15T04:47:18.280 INFO:teuthology.orchestra.run.smithi141.stdout:172.21.15.141 smithi141.front.sepia.ceph.com smithi141 2022-03-15T04:47:18.655 INFO:teuthology.provision.fog.smithi141:Deploy complete! 2022-03-15T04:47:18.814 INFO:teuthology.lock.ops:Checking smithi141.front.sepia.ceph.com 2022-03-15T04:47:18.816 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-15T04:47:18.836 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com 2022-03-15T04:47:18.837 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-15T04:47:18.856 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: set os type and version on server 2022-03-15T04:47:18.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2022-03-15T04:47:19.045 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64 2022-03-15T04:47:19.103 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="CentOS Stream" 2022-03-15T04:47:19.103 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8" 2022-03-15T04:47:19.104 INFO:teuthology.orchestra.run.smithi141.stdout:ID="centos" 2022-03-15T04:47:19.104 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="rhel fedora" 2022-03-15T04:47:19.104 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8" 2022-03-15T04:47:19.104 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8" 2022-03-15T04:47:19.105 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-15T04:47:19.105 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31" 2022-03-15T04:47:19.105 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-15T04:47:19.105 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://centos.org/" 2022-03-15T04:47:19.106 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-15T04:47:19.106 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-15T04:47:19.106 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-15T04:47:19.108 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2022-03-15T04:47:19.127 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server 2022-03-15T04:47:19.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-03-15T04:47:19.328 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2022-03-15T04:47:19.388 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2022-03-15T04:47:19.388 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8" 2022-03-15T04:47:19.388 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2022-03-15T04:47:19.389 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2022-03-15T04:47:19.389 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8" 2022-03-15T04:47:19.389 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2022-03-15T04:47:19.389 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-15T04:47:19.389 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2022-03-15T04:47:19.390 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-15T04:47:19.390 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2022-03-15T04:47:19.390 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-15T04:47:19.390 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-15T04:47:19.391 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-15T04:47:19.391 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-03-15T04:47:19.517 INFO:teuthology.dispatcher.supervisor:Running job 6736013 2022-03-15T04:47:19.518 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6736013 --name vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} -- /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6736013/orig.config.yaml 2022-03-15T04:47:19.524 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6736013 2022-03-15T04:47:19.525 INFO:teuthology.dispatcher.supervisor:Job PID: 17324 2022-03-15T04:47:19.525 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-15T11:17:48.580 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-15T11:17:48.793 INFO:teuthology.kill:Killing Pids: {17324} 2022-03-15T11:17:50.127 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-15T11:17:50.128 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-15T11:17:50.128 INFO:teuthology.misc:Compressing logs... 2022-03-15T11:17:50.129 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2022-03-15T11:17:50.348 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-03-15T11:17:51.823 INFO:teuthology.misc:Compressing logs... 2022-03-15T11:17:52.138 INFO:teuthology.misc:Compressing logs... 2022-03-15T11:27:17.988 INFO:teuthology.kill:No teuthology processes running 2022-03-15T11:27:19.274 INFO:teuthology.kill:Nuking machines: ['smithi141', 'smithi145'] 2022-03-15T11:27:20.317 INFO:teuthology.kill:2022-03-15 11:27:20,317.317 INFO:teuthology.nuke:targets: 2022-03-15T11:27:20.318 INFO:teuthology.kill: smithi141.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmwNJV20FR3MTQb3lsLU4MWVZFU/qbd+Gz/Z3L7SsUkOvowy0UK0n5VYoysvFssQX4UV4U1ooN18wVVp9Z+HVbpbZQjM4NgyJ4jNDUgXcSgsSW5lT2LvzllneAoCvt/Ee4YFk7zQc4r0nb5CGRnbi3e3RGXMqCgsjPjkplL7Gcio+TuErEaR42N+4MFrL2bKHKMO1upOPnNiGZzznUzmjldyWfjKV9mSOpExpJXNWXrFxiXB9Q7wWHzdc0dImJ7SDuh659yXV79oG3Z7wOCZR3r27UAjJdq4hPWxIzcKYQ9dU64jsCfskW/qzrZaFgvGISQTrFwp/lCXRxYu0B4E4JEqyakLCTCLTwX57c4Vqe8hUIPSlqWIkfIKrpS/ogSZ5FP9l5cXfdE69VFhRFU9Mt50rKxinxQE/6GaT7St66IX9cNFvzN37CVdsU3BzszB4cmuo41B2ch8mxpLAmdxu6NTI82Q2FZCyTUq7qozLYgE9NQZzBNRyxrL4N7INITFE= 2022-03-15T11:27:20.318 INFO:teuthology.kill: smithi145.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyL4WoEraY+wl7YEv23f0RF/evHRXeU6d/AWiTI7Ga4sSqHY8udlMVetxwDRSTuyf3x5vz7iTsugPlzDZ+BBS+kkG5mbIvAID5xLjbZ/jlHMd//dKJyXGgvY67dA+5zTrUb64jMJDJj5YI/Y1yW0wTghKAqXgQEvIzHi0025GWBYFXEqP52kAGxh5RtzG6OLcACjMfaJwY99h4s3qg8gUgQ6+kyW8Q/oGM1Q2u2fZQrxDmbBA0zsYU5i13tUIHF2NAG9zD3E5glNf27oyAqjzTpRDu1sI4VQiv7/Wv7TuIVX1Bvn5O+lDCbMXecs252UYn1FHQIidoEbdjew0o83ofrg9DQjMCVTC0rvBBbv074R69L7UStPGU+fXAG6QwkxwBCM48mt7vPiQvpJoTPodvD8kD3kpNgVB4/7eZ9rQfZHaidmZ0IQh9JBjVesJQEBhOPI7UuhQ/qhHgdnKHX8vivw+HRhRwLP0unF6+2iwkgcyTmQ7/aSstl9RQndbfGqs= 2022-03-15T11:27:20.349 INFO:teuthology.kill:2022-03-15 11:27:20,347.347 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-15T11:27:20.356 INFO:teuthology.kill:2022-03-15 11:27:20,355.355 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-15T11:27:20.449 INFO:teuthology.kill:2022-03-15 11:27:20,448.448 INFO:teuthology.orchestra.console:Power off smithi141 2022-03-15T11:27:20.466 INFO:teuthology.kill:2022-03-15 11:27:20,465.465 INFO:teuthology.orchestra.console:Power off smithi145 2022-03-15T11:27:32.947 INFO:teuthology.kill:2022-03-15 11:27:32,946.946 INFO:teuthology.orchestra.console:Power off for smithi141 completed 2022-03-15T11:27:32.959 INFO:teuthology.kill:2022-03-15 11:27:32,959.959 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-03-15T11:27:33.092 INFO:teuthology.kill:2022-03-15 11:27:33,091.091 INFO:teuthology.lock.ops:unlocked: smithi141.front.sepia.ceph.com 2022-03-15T11:27:33.101 INFO:teuthology.kill:2022-03-15 11:27:33,101.101 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com 2022-03-15T11:29:33.434 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-15T11:29:33.549 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi141 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-03-15_11:24:47-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6738231 2022-03-15T11:29:33.550 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi145 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-03-15_11:24:47-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6738231