2022-03-14T10:32:23.420 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-14T10:32:23.563 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging 2022-03-14T10:32:23.565 INFO:teuthology.lock.ops:Updating [ubuntu@smithi154.front.sepia.ceph.com]: reset os type and version on server 2022-03-14T10:32:23.565 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2022-03-14T10:32:23.583 INFO:teuthology.lock.ops:Node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging is complete 2022-03-14T10:32:23.584 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging 2022-03-14T10:32:23.584 INFO:teuthology.lock.ops:Updating [ubuntu@smithi179.front.sepia.ceph.com]: reset os type and version on server 2022-03-14T10:32:23.585 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2022-03-14T10:32:23.605 INFO:teuthology.lock.ops:Node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging is complete 2022-03-14T10:32:23.797 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of centos 8.stream 2022-03-14T10:32:23.815 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of centos 8.stream 2022-03-14T10:32:24.099 INFO:teuthology.orchestra.console:Power off smithi154 2022-03-14T10:32:24.115 INFO:teuthology.orchestra.console:Power off smithi179 2022-03-14T10:32:32.471 INFO:teuthology.orchestra.console:Power off for smithi154 completed 2022-03-14T10:32:32.479 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2022-03-14T10:32:32.573 INFO:teuthology.orchestra.console:Power on smithi154 2022-03-14T10:32:32.582 INFO:teuthology.orchestra.console:Power on smithi179 2022-03-14T10:32:40.942 INFO:teuthology.orchestra.console:Power on for smithi154 completed 2022-03-14T10:32:40.947 INFO:teuthology.orchestra.console:Power on for smithi179 completed 2022-03-14T10:32:41.044 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish 2022-03-14T10:32:41.048 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish 2022-03-14T10:36:47.458 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-14T10:37:12.181 INFO:teuthology.provision.fog.smithi179:Node is ready 2022-03-14T10:37:12.204 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com 2022-03-14T10:37:12.263 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179 2022-03-14T10:37:12.641 INFO:teuthology.provision.fog.smithi179:Deploy complete! 2022-03-14T10:37:37.016 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-14T10:37:37.264 INFO:teuthology.provision.fog.smithi154:Node is ready 2022-03-14T10:37:37.282 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com 2022-03-14T10:37:37.340 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154 2022-03-14T10:37:37.695 INFO:teuthology.provision.fog.smithi154:Deploy complete! 2022-03-14T10:37:37.846 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com 2022-03-14T10:37:37.847 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-14T10:37:37.866 INFO:teuthology.lock.ops:Checking smithi179.front.sepia.ceph.com 2022-03-14T10:37:37.866 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-14T10:37:37.888 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server 2022-03-14T10:37:37.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2022-03-14T10:37:38.135 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2022-03-14T10:37:38.192 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2022-03-14T10:37:38.192 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8" 2022-03-14T10:37:38.192 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2022-03-14T10:37:38.193 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2022-03-14T10:37:38.193 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8" 2022-03-14T10:37:38.193 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8" 2022-03-14T10:37:38.193 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-14T10:37:38.194 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2022-03-14T10:37:38.194 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-14T10:37:38.194 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2022-03-14T10:37:38.195 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-14T10:37:38.195 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-14T10:37:38.196 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-14T10:37:38.197 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2022-03-14T10:37:38.216 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: set os type and version on server 2022-03-14T10:37:38.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2022-03-14T10:37:38.465 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2022-03-14T10:37:38.522 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2022-03-14T10:37:38.522 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8" 2022-03-14T10:37:38.523 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2022-03-14T10:37:38.523 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2022-03-14T10:37:38.523 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8" 2022-03-14T10:37:38.523 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8" 2022-03-14T10:37:38.524 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-14T10:37:38.524 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2022-03-14T10:37:38.524 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-14T10:37:38.524 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2022-03-14T10:37:38.525 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-14T10:37:38.525 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-14T10:37:38.525 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-14T10:37:38.526 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2022-03-14T10:37:38.631 INFO:teuthology.dispatcher.supervisor:Running job 6735773 2022-03-14T10:37:38.631 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735773 --name vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} -- /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735773/orig.config.yaml 2022-03-14T10:37:38.637 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735773 2022-03-14T10:37:38.637 INFO:teuthology.dispatcher.supervisor:Job PID: 17278 2022-03-14T10:37:38.638 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-14T17:08:07.777 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-14T17:08:07.992 INFO:teuthology.kill:Killing Pids: {17278} 2022-03-14T17:08:09.204 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-14T17:08:09.205 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-14T17:08:09.206 INFO:teuthology.misc:Compressing logs... 2022-03-14T17:08:09.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2022-03-14T17:08:09.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2022-03-14T17:08:09.859 INFO:teuthology.misc:Compressing logs... 2022-03-14T17:08:10.094 INFO:teuthology.misc:Compressing logs... 2022-03-14T17:17:23.449 INFO:teuthology.kill:No teuthology processes running 2022-03-14T17:17:24.638 INFO:teuthology.kill:Nuking machines: ['smithi154', 'smithi179'] 2022-03-14T17:17:25.656 INFO:teuthology.kill:2022-03-14 17:17:25,656.656 INFO:teuthology.nuke:targets: 2022-03-14T17:17:25.657 INFO:teuthology.kill: smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDga9yBmdiZ9De8tZHCBXvhS3G2XhBkNFsSooEpxT0Ek6Ov97P+sY5rQSdoJVD70fQgPRHiXe0bcNEb+bfAVLOIBVNp+28TeURaTD3/xcWxx8GbR2J7LwI7CgTnPJOBfZAqxAAw8xWR0wToyidtC6fj3S7rhbSaneDvubSRQhHKR0LIFXwcZ1drh87A0RKwzY6rBzK8fnCmBbEbRh5FgHr1BchcCY1QggnlZGu78D3n5USgRfJFZPOTS/shgOGdxBWVw++hXfBO7eXNYv0pB7LePfjdhYnkjOPLCZMjxQwEJZdjpV9UJrOTl4uQetOEvhI11GsZ8YPBUegOgf6fIvMETMKcx1zR+laVevOqzz4g/dNrZP1DQtfv2tuUIz0j0JxAy44uuagPcax/ISH0iv1vFXLnn6ARkMBQzDA1xPU5Rj1HWzYSNCT4SbT/+U7ImeDkfAxFa6BgYzyCs/elj4/qkBqaZ/w9Nlog1RYIvbC0a2dqGF7dMziGw8Ma5zZ8580= 2022-03-14T17:17:25.658 INFO:teuthology.kill: smithi179.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPcAusO4BGlK5ESCYfHmE/cBK9esDUgELA/eB3xUBPx/HN7vUxwI6lVbrZ4Glhdji0SqoXfk8pffhz2XyeIAbfCpiEOGcz5B4zBq/CwnUOKKJRR5/y2jLemGEBG2JICPkuTDBYJY70iqZuVaTO3JKL5TzXmiJVW9u0CXKUGOBlIkbqoFUaE/PzUKRfkZsAo9h7HI/cs4fvkMC9N5FsOCvBNlUnheoH1HrKPM9M9vka3a4n+Ja2vvTDte0QaFpEpeJLmsofRkk3UXCdhZkdFpr48brcX9QqJe/j+o1WLBE5JuiiwO6O82VHbJkubPWBHS1Tj/TbTVEMGCvjI7VAN/HXWaWdSUGJxxUaqnUdRojiHVblPkmxTtAVM7BtlvkMXczTSAGHsBZsj4qLP4I0WKJY1/l2yLIKLEYKOGKa3cBhv0K9yohqz9ymxn9rVOVrWhTCiDZEFwy2PKc9AmkpN8LPHFotD0dNM6TpjIRV+qjZ5AV1uPUuHeKdhvMXX7lqPc8= 2022-03-14T17:17:25.684 INFO:teuthology.kill:2022-03-14 17:17:25,683.683 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-14T17:17:25.687 INFO:teuthology.kill:2022-03-14 17:17:25,687.687 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-14T17:17:25.765 INFO:teuthology.kill:2022-03-14 17:17:25,765.765 INFO:teuthology.orchestra.console:Power off smithi154 2022-03-14T17:17:25.779 INFO:teuthology.kill:2022-03-14 17:17:25,779.779 INFO:teuthology.orchestra.console:Power off smithi179 2022-03-14T17:17:34.139 INFO:teuthology.kill:2022-03-14 17:17:34,139.139 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2022-03-14T17:17:34.293 INFO:teuthology.kill:2022-03-14 17:17:34,292.292 INFO:teuthology.lock.ops:unlocked: smithi179.front.sepia.ceph.com 2022-03-14T17:17:38.244 INFO:teuthology.kill:2022-03-14 17:17:38,243.243 INFO:teuthology.orchestra.console:Power off for smithi154 completed 2022-03-14T17:17:38.390 INFO:teuthology.kill:2022-03-14 17:17:38,390.390 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com 2022-03-14T17:19:38.687 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-14T17:19:38.834 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi154 but it was locked by another job: /home/teuthworker/archive/rfriedma-2022-03-14_16:55:43-rados-wip-rf-42714-pacific-distro-basic-smithi/6736130