2022-03-20T05:52:35.470 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-20T05:52:35.595 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi019.front.sepia.ceph.com' reimaging 2022-03-20T05:52:35.596 INFO:teuthology.lock.ops:Updating [ubuntu@smithi019.front.sepia.ceph.com]: reset os type and version on server 2022-03-20T05:52:35.597 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2022-03-20T05:52:35.618 INFO:teuthology.lock.ops:Node 'ubuntu@smithi019.front.sepia.ceph.com' reimaging is complete 2022-03-20T05:52:35.618 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging 2022-03-20T05:52:35.619 INFO:teuthology.lock.ops:Updating [ubuntu@smithi092.front.sepia.ceph.com]: reset os type and version on server 2022-03-20T05:52:35.620 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2022-03-20T05:52:35.639 INFO:teuthology.lock.ops:Node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging is complete 2022-03-20T05:52:35.820 INFO:teuthology.provision.fog.smithi019:Scheduling deploy of centos 8.stream 2022-03-20T05:52:35.833 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of centos 8.stream 2022-03-20T05:52:36.175 INFO:teuthology.orchestra.console:Power off smithi019 2022-03-20T05:52:36.236 INFO:teuthology.orchestra.console:Power off smithi092 2022-03-20T05:52:48.660 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2022-03-20T05:52:48.708 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2022-03-20T05:52:48.762 INFO:teuthology.orchestra.console:Power on smithi019 2022-03-20T05:52:48.809 INFO:teuthology.orchestra.console:Power on smithi092 2022-03-20T05:52:53.016 INFO:teuthology.orchestra.console:Power on for smithi019 completed 2022-03-20T05:52:53.059 INFO:teuthology.orchestra.console:Power on for smithi092 completed 2022-03-20T05:52:53.118 INFO:teuthology.provision.fog.smithi019:Waiting for deploy to finish 2022-03-20T05:52:53.160 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish 2022-03-20T05:57:16.105 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-20T05:57:16.366 INFO:teuthology.provision.fog.smithi019:Node is ready 2022-03-20T05:57:16.385 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019.front.sepia.ceph.com 2022-03-20T05:57:16.444 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19 smithi019.front.sepia.ceph.com smithi019 2022-03-20T05:57:16.806 INFO:teuthology.provision.fog.smithi019:Deploy complete! 2022-03-20T05:57:31.080 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-20T05:57:31.316 INFO:teuthology.provision.fog.smithi092:Node is ready 2022-03-20T05:57:31.336 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com 2022-03-20T05:57:31.395 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092 2022-03-20T05:57:31.790 INFO:teuthology.provision.fog.smithi092:Deploy complete! 2022-03-20T05:57:32.000 INFO:teuthology.lock.ops:Checking smithi019.front.sepia.ceph.com 2022-03-20T05:57:32.001 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-20T05:57:32.029 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com 2022-03-20T05:57:32.030 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-20T05:57:32.048 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: set os type and version on server 2022-03-20T05:57:32.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2022-03-20T05:57:32.299 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2022-03-20T05:57:32.358 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2022-03-20T05:57:32.358 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="8" 2022-03-20T05:57:32.358 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2022-03-20T05:57:32.359 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2022-03-20T05:57:32.359 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="8" 2022-03-20T05:57:32.360 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el8" 2022-03-20T05:57:32.360 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-20T05:57:32.360 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2022-03-20T05:57:32.360 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-20T05:57:32.361 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2022-03-20T05:57:32.361 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-20T05:57:32.361 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-20T05:57:32.361 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-20T05:57:32.363 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2022-03-20T05:57:32.382 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server 2022-03-20T05:57:32.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2022-03-20T05:57:32.575 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2022-03-20T05:57:32.633 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2022-03-20T05:57:32.633 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8" 2022-03-20T05:57:32.633 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2022-03-20T05:57:32.634 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2022-03-20T05:57:32.634 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8" 2022-03-20T05:57:32.634 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2022-03-20T05:57:32.634 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-20T05:57:32.635 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2022-03-20T05:57:32.635 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-20T05:57:32.635 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2022-03-20T05:57:32.635 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-20T05:57:32.636 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-20T05:57:32.636 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-20T05:57:32.636 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2022-03-20T05:57:32.731 INFO:teuthology.dispatcher.supervisor:Running job 6747576 2022-03-20T05:57:32.731 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747576 --name vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} -- /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747576/orig.config.yaml 2022-03-20T05:57:32.734 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747576 2022-03-20T05:57:32.736 INFO:teuthology.dispatcher.supervisor:Job PID: 22692 2022-03-20T05:57:32.736 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-20T12:28:01.125 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-20T12:28:01.287 INFO:teuthology.kill:Killing Pids: {22692} 2022-03-20T12:28:02.426 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-20T12:28:02.427 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-20T12:28:02.427 INFO:teuthology.misc:Compressing logs... 2022-03-20T12:28:02.428 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2022-03-20T12:28:02.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2022-03-20T12:28:03.109 INFO:teuthology.misc:Compressing logs... 2022-03-20T12:28:03.360 INFO:teuthology.misc:Compressing logs... 2022-03-20T12:38:37.438 INFO:teuthology.kill:No teuthology processes running 2022-03-20T12:38:38.562 INFO:teuthology.kill:Nuking machines: ['smithi019', 'smithi092'] 2022-03-20T12:38:39.525 INFO:teuthology.kill:2022-03-20 12:38:39,524.524 INFO:teuthology.nuke:targets: 2022-03-20T12:38:39.526 INFO:teuthology.kill: smithi019.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJyey+FfnZTW1ON3KAWJNEEMPGmO20aiLz5jzgoFtClz+qP14zcRnM2mJITATf4CtQQYq+Dg/biRZzemQtlIsZaFTebM5CerXsZvloeQbVEjA+2FLAoeinPmZ93WY3dA4pC+3QoDvR7Q/0WmNSw8hSmnt01x0RqPKS694x5crlQq3ANNOS2/nRoTsMOOfctWFmwisxc9+8RMotcg7oYOHpMOT2NXDc2K+gk5JsS/hPUS1yCrDQadTX5sn49LidEnG/shGXWr4mZBNMZyq/maxw4LBXhjX1zltj4Mi50arcUsoXXdN3rxOym7DZNplvpVuTQKiHzc5bFCn5Fz37vmCDtrlFMXGWPLUm6lRXIsT4O5Vekq0lUxZtiiuVtEmewchbFeXHlMaUXFZZrAu5ICBDk6rKSz86Mh0nSg7AlOgofywGPyULFppChV7l23uGmiy65VlWdY4OD98r2lGJ7ZvweX86e3Q8/bymdU1m8t5hgLXhhLmNEjD3HARdam/++HU= 2022-03-20T12:38:39.526 INFO:teuthology.kill: smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCeBhhkrM/kAxYOb5WWiH7VKemtHBMzp/DF34iH+r7HpxwBM7H6io7AYnh2PNqhombF88Kov6wciXIh/PL3MhK20vCUObvqU0NfNf+tBzOxzgvt1hHH4U3slbx6cfNnY3YggkmV5mtA6dm1fxVIbKlPnILSBTLn37yMf7PnkJr0RhbwwXsx6R0MlgdRS7gkTEUjjA1PqfKHrt9+H68pCiog5KYeww1+5WGI3V8PdvG24yJryoO6TM2pOuePjTmesXNnymM6IIvj54i9LiHiYrtfQs+7wM2x6xVoM7upvMFMc542M76+jybSnU/iGn0gIQ4zHK9/tfPgEeYKkCrlRail/3xJ6gCdBam8orLMMZzKBbqKvq4JIJnJQFGT7UJAjcP2uyMrvhvLq1cOWP4pfouwV03275hATp4pHaJoV6It259bWWTGA7sySICYVoS39yULTKrMEr5APh+fyAO6XRuH+9VWduFH90c7zsdafauBujN8A8BTBGb7Sj3v8E/LEZE= 2022-03-20T12:38:39.551 INFO:teuthology.kill:2022-03-20 12:38:39,551.551 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-20T12:38:39.554 INFO:teuthology.kill:2022-03-20 12:38:39,553.553 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-20T12:38:39.628 INFO:teuthology.kill:2022-03-20 12:38:39,627.627 INFO:teuthology.orchestra.console:Power off smithi019 2022-03-20T12:38:39.640 INFO:teuthology.kill:2022-03-20 12:38:39,639.639 INFO:teuthology.orchestra.console:Power off smithi092 2022-03-20T12:38:47.992 INFO:teuthology.kill:2022-03-20 12:38:47,992.992 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2022-03-20T12:38:48.136 INFO:teuthology.kill:2022-03-20 12:38:48,135.135 INFO:teuthology.lock.ops:unlocked: smithi019.front.sepia.ceph.com 2022-03-20T12:38:52.109 INFO:teuthology.kill:2022-03-20 12:38:52,109.109 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2022-03-20T12:38:52.252 INFO:teuthology.kill:2022-03-20 12:38:52,252.252 INFO:teuthology.lock.ops:unlocked: smithi092.front.sepia.ceph.com 2022-03-20T12:40:52.529 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-20T12:40:52.673 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi019 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747777 2022-03-20T12:40:52.673 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi092 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-20_02:16:37-fs-wip-vshankar-testing-20220319-163539-testing-default-smithi/6747777