2023-03-03T05:20:07.093 INFO:root:teuthology version: 0.0.1.dev49+gc7c3cc8
2023-03-03T05:20:07.236 INFO:teuthology.lock.ops:Start node 'smithi102.front.sepia.ceph.com' reimaging
2023-03-03T05:20:07.238 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: reset os type and version on server
2023-03-03T05:20:07.238 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server
2023-03-03T05:20:07.262 INFO:teuthology.lock.ops:Node 'smithi102.front.sepia.ceph.com' reimaging is complete
2023-03-03T05:20:07.262 INFO:teuthology.lock.ops:Start node 'smithi172.front.sepia.ceph.com' reimaging
2023-03-03T05:20:07.263 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: reset os type and version on server
2023-03-03T05:20:07.263 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server
2023-03-03T05:20:07.286 INFO:teuthology.lock.ops:Node 'smithi172.front.sepia.ceph.com' reimaging is complete
2023-03-03T05:20:07.439 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 8.stream
2023-03-03T05:20:07.443 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of centos 8.stream
2023-03-03T05:20:07.715 INFO:teuthology.orchestra.console:Power off smithi172
2023-03-03T05:20:07.731 INFO:teuthology.orchestra.console:Power off smithi102
2023-03-03T05:20:16.087 INFO:teuthology.orchestra.console:Power off for smithi172 completed
2023-03-03T05:20:16.095 INFO:teuthology.orchestra.console:Power off for smithi102 completed
2023-03-03T05:20:16.188 INFO:teuthology.orchestra.console:Power on smithi172
2023-03-03T05:20:16.198 INFO:teuthology.orchestra.console:Power on smithi102
2023-03-03T05:20:24.561 INFO:teuthology.orchestra.console:Power on for smithi102 completed
2023-03-03T05:20:24.652 INFO:teuthology.orchestra.console:Power on for smithi172 completed
2023-03-03T05:20:24.663 INFO:teuthology.provision.fog.smithi102:Waiting for deploy to finish
2023-03-03T05:20:24.754 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish
2023-03-03T05:23:13.701 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172
2023-03-03T05:23:13.702 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-03T05:23:22.793 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172
2023-03-03T05:23:22.793 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-03T05:23:31.877 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172
2023-03-03T05:23:31.878 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-03T05:23:40.933 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172
2023-03-03T05:23:40.933 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-03T05:23:49.989 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172
2023-03-03T05:23:49.990 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-03T05:23:59.045 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172
2023-03-03T05:23:59.045 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-03T05:24:08.101 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172
2023-03-03T05:24:17.189 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.172
2023-03-03T05:24:23.429 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-03T05:24:32.485 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.102
2023-03-03T05:24:39.882 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-03T05:24:55.116 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-03T05:25:04.776 INFO:teuthology.provision.fog.smithi102:Node is ready
2023-03-03T05:25:04.798 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102.front.sepia.ceph.com
2023-03-03T05:25:04.857 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102 smithi102.front.sepia.ceph.com smithi102
2023-03-03T05:25:05.230 INFO:teuthology.provision.fog.smithi102:Deploy complete!
2023-03-03T05:25:15.565 INFO:teuthology.provision.fog.smithi172:Node is ready
2023-03-03T05:25:15.583 INFO:teuthology.orchestra.run.smithi172.stdout:smithi001
2023-03-03T05:25:15.641 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.1 smithi001 smithi001
2023-03-03T05:25:16.013 INFO:teuthology.provision.fog.smithi172:Deploy complete!
2023-03-03T05:25:16.187 INFO:teuthology.lock.ops:Checking smithi102.front.sepia.ceph.com
2023-03-03T05:25:16.188 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-03T05:25:16.208 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com
2023-03-03T05:25:16.208 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-03T05:25:16.226 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: set os type and version on server
2023-03-03T05:25:16.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-03T05:25:16.494 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64
2023-03-03T05:25:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream"
2023-03-03T05:25:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8"
2023-03-03T05:25:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos"
2023-03-03T05:25:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora"
2023-03-03T05:25:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8"
2023-03-03T05:25:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8"
2023-03-03T05:25:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-03T05:25:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31"
2023-03-03T05:25:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-03T05:25:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/"
2023-03-03T05:25:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-03T05:25:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-03T05:25:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-03T05:25:16.558 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server
2023-03-03T05:25:16.584 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server
2023-03-03T05:25:16.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com'
2023-03-03T05:25:16.777 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64
2023-03-03T05:25:16.835 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream"
2023-03-03T05:25:16.836 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8"
2023-03-03T05:25:16.836 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos"
2023-03-03T05:25:16.836 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora"
2023-03-03T05:25:16.836 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8"
2023-03-03T05:25:16.836 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8"
2023-03-03T05:25:16.836 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-03T05:25:16.836 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31"
2023-03-03T05:25:16.837 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-03T05:25:16.837 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/"
2023-03-03T05:25:16.837 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-03T05:25:16.837 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-03T05:25:16.837 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-03T05:25:16.838 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server
2023-03-03T05:25:16.948 INFO:teuthology.dispatcher.supervisor:Running job 7192093
2023-03-03T05:25:16.949 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192093 --name vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} -- /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192093/orig.config.yaml
2023-03-03T05:25:16.952 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192093
2023-03-03T05:25:16.953 INFO:teuthology.dispatcher.supervisor:Job PID: 28878
2023-03-03T05:25:16.953 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-03T17:25:41.151 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-03T17:25:41.324 INFO:teuthology.kill:Killing Pids: {28878}
2023-03-03T17:25:42.187 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2023-03-03T17:25:42.188 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2023-03-03T17:25:42.188 INFO:teuthology.misc:Compressing logs...
2023-03-03T17:25:42.189 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2023-03-03T17:25:42.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com'
2023-03-03T17:25:42.808 INFO:teuthology.misc:Compressing logs...
2023-03-03T17:25:43.109 INFO:teuthology.misc:Compressing logs...
2023-03-03T17:27:35.305 INFO:teuthology.kill:No teuthology processes running
2023-03-03T17:27:36.104 INFO:teuthology.kill:Nuking machines: ['smithi102', 'smithi172']
2023-03-03T17:27:36.716 INFO:teuthology.kill:2023-03-03 17:27:36,716.716 INFO:teuthology.nuke:targets:
2023-03-03T17:27:36.718 INFO:teuthology.kill:  smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJOZkGy1FXGwI7zar+JgG/pI7NbBPl0NOU0x42EcareR46Y0bdu2QAQdWacApFIsIq4S32kMe+audUeP0nlFn4Y=
2023-03-03T17:27:36.718 INFO:teuthology.kill:  smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1SyCVoQPMW4XOWT/PxlpBz+odYm8m6q9lBogugRDxqVJ5FiaELoLATbj70I+3K/pcqEXXmsrIxYFalyDvRhIqJ5tUsqy/rinKy/S6kyX9ESiUmKWQHEOHe2Yyoy5YrjH18+LG9bgYyR8nI+0ifqdEJ0+t+nzPkN50DDJMPYoSdB7tvCJDeLNz8vOJ9nX7jqtTyWg2dtSFW/Ah0YoOTDieL3p67NoJ0Gf6uSHR8p0J0to4hC90yY/UAG4EQBtOMM4ZN/0DdPnKyQvuN9sGQ+XpTqQ+ok0HUulGbn0oEaovOL7glS1eipn+aqV0BNdlyldxTjQju5kdw7KVL6WFrH1cydV7KAnT04x6uv7dCGFj5199v2HZg47aAj9AYV7K08JUUsbPPiL7WolOXjcJbUAvLVlyrNDPoxGOtPYv0Fnq2oNCj6bK6IoN0vH0lXipVf4BjdIYxXxaPS9GXZGV2iQ2anuGvi2HBJ+KlYFSJgrqYFLLH6dqmyRApgmGknw1XcU=
2023-03-03T17:27:36.718 INFO:teuthology.kill:2023-03-03 17:27:36,716.716 INFO:teuthology.nuke:Checking targets against current locks
2023-03-03T17:27:36.782 INFO:teuthology.kill:2023-03-03 17:27:36,781.781 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-03T17:27:36.804 INFO:teuthology.kill:2023-03-03 17:27:36,804.804 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-03T17:27:36.864 INFO:teuthology.kill:2023-03-03 17:27:36,864.864 INFO:teuthology.orchestra.console:Power off smithi102
2023-03-03T17:27:36.913 INFO:teuthology.kill:2023-03-03 17:27:36,912.912 INFO:teuthology.orchestra.console:Power off smithi172
2023-03-03T17:27:49.360 INFO:teuthology.kill:2023-03-03 17:27:49,360.360 INFO:teuthology.orchestra.console:Power off for smithi102 completed
2023-03-03T17:27:49.394 INFO:teuthology.kill:2023-03-03 17:27:49,394.394 INFO:teuthology.orchestra.console:Power off for smithi172 completed
2023-03-03T17:27:49.525 INFO:teuthology.kill:2023-03-03 17:27:49,525.525 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com
2023-03-03T17:27:49.534 INFO:teuthology.kill:2023-03-03 17:27:49,533.533 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com
2023-03-03T17:29:49.957 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-03T17:29:50.067 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi102 but it was locked by another job: /home/teuthworker/archive/rishabh-2023-03-03_17:19:21-fs:multifs-wip-rishabh-fs-auth-subcmd-2023Feb28-2023Mar02-1948-distro-default-smithi/7192470
2023-03-03T17:29:50.068 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi172 but it was locked by another job: /home/teuthworker/archive/rishabh-2023-03-03_17:19:21-fs:multifs-wip-rishabh-fs-auth-subcmd-2023Feb28-2023Mar02-1948-distro-default-smithi/7192470