2022-03-14T12:18:10.363 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-14T12:18:10.499 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging 2022-03-14T12:18:10.500 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server 2022-03-14T12:18:10.500 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2022-03-14T12:18:10.519 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete 2022-03-14T12:18:10.520 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi133.front.sepia.ceph.com' reimaging 2022-03-14T12:18:10.520 INFO:teuthology.lock.ops:Updating [ubuntu@smithi133.front.sepia.ceph.com]: reset os type and version on server 2022-03-14T12:18:10.520 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-03-14T12:18:10.539 INFO:teuthology.lock.ops:Node 'ubuntu@smithi133.front.sepia.ceph.com' reimaging is complete 2022-03-14T12:18:10.733 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of ubuntu 20.04 2022-03-14T12:18:10.741 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of ubuntu 20.04 2022-03-14T12:18:11.032 INFO:teuthology.orchestra.console:Power off smithi116 2022-03-14T12:18:11.046 INFO:teuthology.orchestra.console:Power off smithi133 2022-03-14T12:18:11.176 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2022-03-14T12:18:11.186 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2022-03-14T12:18:11.278 INFO:teuthology.orchestra.console:Power on smithi116 2022-03-14T12:18:11.287 INFO:teuthology.orchestra.console:Power on smithi133 2022-03-14T12:18:19.639 INFO:teuthology.orchestra.console:Power on for smithi116 completed 2022-03-14T12:18:19.732 INFO:teuthology.orchestra.console:Power on for smithi133 completed 2022-03-14T12:18:19.741 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2022-03-14T12:18:19.833 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish 2022-03-14T12:23:16.961 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-14T12:23:17.411 INFO:teuthology.provision.fog.smithi133:Node is ready 2022-03-14T12:23:17.418 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com 2022-03-14T12:23:17.425 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133 2022-03-14T12:23:17.588 INFO:teuthology.provision.fog.smithi133:Deploy complete! 2022-03-14T12:23:18.198 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-14T12:23:18.602 INFO:teuthology.provision.fog.smithi116:Node is ready 2022-03-14T12:23:18.608 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2022-03-14T12:23:18.616 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2022-03-14T12:23:18.784 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2022-03-14T12:23:18.984 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2022-03-14T12:23:18.985 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-14T12:23:19.003 INFO:teuthology.lock.ops:Checking smithi133.front.sepia.ceph.com 2022-03-14T12:23:19.004 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-14T12:23:19.024 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2022-03-14T12:23:19.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2022-03-14T12:23:19.239 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2022-03-14T12:23:19.246 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Ubuntu" 2022-03-14T12:23:19.247 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-14T12:23:19.247 INFO:teuthology.orchestra.run.smithi116.stdout:ID=ubuntu 2022-03-14T12:23:19.247 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE=debian 2022-03-14T12:23:19.247 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-14T12:23:19.248 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="20.04" 2022-03-14T12:23:19.248 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-14T12:23:19.248 INFO:teuthology.orchestra.run.smithi116.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-14T12:23:19.248 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-14T12:23:19.249 INFO:teuthology.orchestra.run.smithi116.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-14T12:23:19.249 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_CODENAME=focal 2022-03-14T12:23:19.249 INFO:teuthology.orchestra.run.smithi116.stdout:UBUNTU_CODENAME=focal 2022-03-14T12:23:19.250 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2022-03-14T12:23:19.270 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: set os type and version on server 2022-03-14T12:23:19.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-03-14T12:23:19.442 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2022-03-14T12:23:19.448 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Ubuntu" 2022-03-14T12:23:19.448 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-14T12:23:19.449 INFO:teuthology.orchestra.run.smithi133.stdout:ID=ubuntu 2022-03-14T12:23:19.449 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE=debian 2022-03-14T12:23:19.449 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-14T12:23:19.449 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="20.04" 2022-03-14T12:23:19.450 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-14T12:23:19.450 INFO:teuthology.orchestra.run.smithi133.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-14T12:23:19.450 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-14T12:23:19.450 INFO:teuthology.orchestra.run.smithi133.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-14T12:23:19.450 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_CODENAME=focal 2022-03-14T12:23:19.451 INFO:teuthology.orchestra.run.smithi133.stdout:UBUNTU_CODENAME=focal 2022-03-14T12:23:19.451 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-03-14T12:23:19.551 INFO:teuthology.dispatcher.supervisor:Running job 6735847 2022-03-14T12:23:19.551 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735847 --name vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} -- /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735847/orig.config.yaml 2022-03-14T12:23:19.555 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-03-14_08:09:40-fs-wip-vshankar-testing1-20220314-103400-testing-default-smithi/6735847 2022-03-14T12:23:19.555 INFO:teuthology.dispatcher.supervisor:Job PID: 28412 2022-03-14T12:23:19.556 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-14T18:53:48.206 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-14T18:53:48.397 INFO:teuthology.kill:Killing Pids: {28412} 2022-03-14T18:53:49.557 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-03-14T18:53:49.558 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-03-14T18:53:49.558 INFO:teuthology.misc:Compressing logs... 2022-03-14T18:53:49.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2022-03-14T18:53:49.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-03-14T18:53:50.249 INFO:teuthology.misc:Compressing logs... 2022-03-14T18:53:50.400 INFO:teuthology.misc:Compressing logs... 2022-03-14T18:56:04.906 INFO:teuthology.kill:No teuthology processes running 2022-03-14T18:56:06.104 INFO:teuthology.kill:Nuking machines: ['smithi116', 'smithi133'] 2022-03-14T18:56:07.108 INFO:teuthology.kill:2022-03-14 18:56:07,108.108 INFO:teuthology.nuke:targets: 2022-03-14T18:56:07.109 INFO:teuthology.kill: smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsGLsQq4i8+iAoMKuROzb/OvC4/P67Fb0wKZiZqaSN676PvnaxS89uq+pYwgKJ8s1ImuF09MNarwRBGfn2ynfJx1UOr1H5/gjRbUnPafRL3xnvxarw9EwJH7BieorcDPrbBFMGSXvyOOThWfKtH7LLndqAEkHUQDQSF3ttHtXm8hJeUsALry58nc1NPaeJxDAQ6F9iDDIfkfa7kDoGhoj9P9euusiWtyR9vgC/azvi40Vn1REhRbCvvcIvf8ru/PzJ28or1B3aWPpihE+ttFA3ABohkKuzQP5khXTQKdYFPCuHovTVnrQwLT3X3iN387/odREQe+cxova3wskWGHDDiA+pGFNtDpIFtKbGfCuGj2FeqpR5hYE+cT34Chbj/AkCbwRffW5QnEnh2d8tyUcan1evF5HPpv8k+GXDfVfI+TM7vRdFu6DbAI5KXCKRj5AkaUyPluuQ2OwjRt4W+IK1CYhDpVWri0njsPXhxS/LXW7+GAGU56VmDt/NVrLGTuk= 2022-03-14T18:56:07.110 INFO:teuthology.kill: smithi133.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUdnpINpdXLpEI3T/Da9zRrx/81D3C5moFIqbTrYn7dWUQVy12NbTubIJwJQ0aZkP2qyGUXzrX0JO2AWoZtf7Rx5i4xrSz0lO12cww1BLLbak1wNQJtbfSiGMro5kEJ30w5UvfTK8ygqcNruangnHudwBgRSDq01vLWMLusCpDLwFRH4pmuOnb4IwEklmpYvGUgWr39JxCvt6pdOmGBVpLl9BTpcxnyx5xeLUdht53dYhN5B+FSzai+VAWXjHYwxY97/lVEgiHJDGmFbtT6iG/5+iFPqLk2bL2S5l8U8apYd+cXmHEcSHODdPXrpZ5wZBQV85LsHOHV965KnMimeW5/My5hd1aNX78gyPR25JPtk6q3q7oAG5OO1AthhVhDGdoyBLVoLCyi/KMbeFKPJ2J6uEUKSsjoExvocFfl8sUxAxpiUts4iu2eJg6jGVy/LMzvuaTCQ6cblvY2HvAHU5QRnXqTt1fURY6cBl06jXgAZ7bTw1o1He7VUIFhrLXFZs= 2022-03-14T18:56:07.134 INFO:teuthology.kill:2022-03-14 18:56:07,134.134 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-14T18:56:07.137 INFO:teuthology.kill:2022-03-14 18:56:07,137.137 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-14T18:56:07.212 INFO:teuthology.kill:2022-03-14 18:56:07,212.212 INFO:teuthology.orchestra.console:Power off smithi133 2022-03-14T18:56:07.226 INFO:teuthology.kill:2022-03-14 18:56:07,226.226 INFO:teuthology.orchestra.console:Power off smithi116 2022-03-14T18:56:19.701 INFO:teuthology.kill:2022-03-14 18:56:19,700.700 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2022-03-14T18:56:19.706 INFO:teuthology.kill:2022-03-14 18:56:19,706.706 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2022-03-14T18:56:19.851 INFO:teuthology.kill:2022-03-14 18:56:19,850.850 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com 2022-03-14T18:56:19.887 INFO:teuthology.kill:2022-03-14 18:56:19,886.886 INFO:teuthology.lock.ops:unlocked: smithi133.front.sepia.ceph.com 2022-03-14T18:58:20.168 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-14T18:58:20.286 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi116 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-14_18:47:44-rados-wip-yuri3-testing-2022-03-14-0946-distro-default-smithi/6736160 2022-03-14T18:58:20.286 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi133 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-14_18:47:44-rados-wip-yuri3-testing-2022-03-14-0946-distro-default-smithi/6736160