2022-10-20T16:52:40.995 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9
2022-10-20T16:52:41.145 INFO:teuthology.lock.ops:Start node 'smithi006.front.sepia.ceph.com' reimaging
2022-10-20T16:52:41.147 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: reset os type and version on server
2022-10-20T16:52:41.147 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server
2022-10-20T16:52:41.167 INFO:teuthology.lock.ops:Node 'smithi006.front.sepia.ceph.com' reimaging is complete
2022-10-20T16:52:41.168 INFO:teuthology.lock.ops:Start node 'smithi018.front.sepia.ceph.com' reimaging
2022-10-20T16:52:41.169 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: reset os type and version on server
2022-10-20T16:52:41.169 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server
2022-10-20T16:52:41.194 INFO:teuthology.lock.ops:Node 'smithi018.front.sepia.ceph.com' reimaging is complete
2022-10-20T16:52:41.388 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of rhel 8.6
2022-10-20T16:52:41.404 INFO:teuthology.provision.fog.smithi018:Scheduling deploy of rhel 8.6
2022-10-20T16:52:41.712 INFO:teuthology.orchestra.console:Power off smithi006
2022-10-20T16:52:41.753 INFO:teuthology.orchestra.console:Power off smithi018
2022-10-20T16:52:41.864 INFO:teuthology.orchestra.console:Power off for smithi006 completed
2022-10-20T16:52:41.909 INFO:teuthology.orchestra.console:Power off for smithi018 completed
2022-10-20T16:52:41.966 INFO:teuthology.orchestra.console:Power on smithi006
2022-10-20T16:52:42.010 INFO:teuthology.orchestra.console:Power on smithi018
2022-10-20T16:52:46.266 INFO:teuthology.orchestra.console:Power on for smithi018 completed
2022-10-20T16:52:46.367 INFO:teuthology.provision.fog.smithi018:Waiting for deploy to finish
2022-10-20T16:52:50.343 INFO:teuthology.orchestra.console:Power on for smithi006 completed
2022-10-20T16:52:50.444 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish
2022-10-20T16:55:02.290 ERROR:teuthology.orchestra.connection:Error authenticating with smithi018.front.sepia.ceph.com: Authentication failed.
2022-10-20T16:56:33.880 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-10-20T16:56:48.359 INFO:teuthology.provision.fog.smithi018:Node is ready
2022-10-20T16:56:48.378 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018.front.sepia.ceph.com
2022-10-20T16:56:48.435 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18 smithi018.front.sepia.ceph.com smithi018
2022-10-20T16:56:48.818 INFO:teuthology.provision.fog.smithi018:Deploy complete!
2022-10-20T16:57:04.142 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-10-20T16:57:04.379 INFO:teuthology.provision.fog.smithi006:Node is ready
2022-10-20T16:57:04.398 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com
2022-10-20T16:57:04.457 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006
2022-10-20T16:57:04.803 INFO:teuthology.provision.fog.smithi006:Deploy complete!
2022-10-20T16:57:05.041 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com
2022-10-20T16:57:05.042 INFO:teuthology.lock.ops:New key found. Updating...
2022-10-20T16:57:05.063 INFO:teuthology.lock.ops:Checking smithi018.front.sepia.ceph.com
2022-10-20T16:57:05.063 INFO:teuthology.lock.ops:New key found. Updating...
2022-10-20T16:57:05.084 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server
2022-10-20T16:57:05.085 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com'
2022-10-20T16:57:05.287 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64
2022-10-20T16:57:05.344 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Red Hat Enterprise Linux"
2022-10-20T16:57:05.344 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8.6 (Ootpa)"
2022-10-20T16:57:05.344 INFO:teuthology.orchestra.run.smithi006.stdout:ID="rhel"
2022-10-20T16:57:05.344 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="fedora"
2022-10-20T16:57:05.345 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8.6"
2022-10-20T16:57:05.345 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8"
2022-10-20T16:57:05.345 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-10-20T16:57:05.345 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31"
2022-10-20T16:57:05.346 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-10-20T16:57:05.346 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.redhat.com/"
2022-10-20T16:57:05.346 INFO:teuthology.orchestra.run.smithi006.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-10-20T16:57:05.346 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-10-20T16:57:05.347 INFO:teuthology.orchestra.run.smithi006.stdout:
2022-10-20T16:57:05.347 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-10-20T16:57:05.347 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-10-20T16:57:05.347 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-10-20T16:57:05.348 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-10-20T16:57:05.349 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server
2022-10-20T16:57:05.376 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: set os type and version on server
2022-10-20T16:57:05.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com'
2022-10-20T16:57:05.591 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64
2022-10-20T16:57:05.651 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="Red Hat Enterprise Linux"
2022-10-20T16:57:05.652 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="8.6 (Ootpa)"
2022-10-20T16:57:05.652 INFO:teuthology.orchestra.run.smithi018.stdout:ID="rhel"
2022-10-20T16:57:05.652 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="fedora"
2022-10-20T16:57:05.653 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="8.6"
2022-10-20T16:57:05.653 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el8"
2022-10-20T16:57:05.653 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-10-20T16:57:05.653 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31"
2022-10-20T16:57:05.653 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-10-20T16:57:05.654 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://www.redhat.com/"
2022-10-20T16:57:05.654 INFO:teuthology.orchestra.run.smithi018.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-10-20T16:57:05.654 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-10-20T16:57:05.654 INFO:teuthology.orchestra.run.smithi018.stdout:
2022-10-20T16:57:05.655 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-10-20T16:57:05.655 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-10-20T16:57:05.655 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-10-20T16:57:05.655 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-10-20T16:57:05.656 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server
2022-10-20T16:57:05.823 INFO:teuthology.dispatcher.supervisor:Running job 7075441
2022-10-20T16:57:05.824 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-10-20_10:31:38-fs-wip-vshankar-testing1-20221017-112130-testing-default-smithi/7075441 --name vshankar-2022-10-20_10:31:38-fs-wip-vshankar-testing1-20221017-112130-testing-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} -- /home/teuthworker/archive/vshankar-2022-10-20_10:31:38-fs-wip-vshankar-testing1-20221017-112130-testing-default-smithi/7075441/orig.config.yaml
2022-10-20T16:57:05.829 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-10-20_10:31:38-fs-wip-vshankar-testing1-20221017-112130-testing-default-smithi/7075441
2022-10-20T16:57:05.830 INFO:teuthology.dispatcher.supervisor:Job PID: 3227
2022-10-20T16:57:05.830 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-10-21T04:57:27.839 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-10-21T04:57:28.232 INFO:teuthology.kill:Killing Pids: {3227}
2022-10-21T04:57:29.571 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-10-21T04:57:29.572 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']
2022-10-21T04:57:29.572 INFO:teuthology.misc:Compressing logs...
2022-10-21T04:57:29.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com'
2022-10-21T04:57:29.843 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com'
2022-10-21T04:57:30.445 INFO:teuthology.misc:Compressing logs...
2022-10-21T04:57:31.743 INFO:teuthology.misc:Compressing logs...
2022-10-21T05:03:46.378 INFO:teuthology.kill:No teuthology processes running
2022-10-21T05:03:47.601 INFO:teuthology.kill:Nuking machines: ['smithi006', 'smithi018']
2022-10-21T05:03:48.579 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-10-21T05:03:48.580 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-10-21T05:03:48.580 INFO:teuthology.kill:2022-10-21 05:03:48,579.579 INFO:teuthology.nuke:targets:
2022-10-21T05:03:48.580 INFO:teuthology.kill:  smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtAuhdSa1y4TEqIgDBOCf0KC8IcKMriciAGBiD7Yh764KjRruWlwuPbyNRDyTArsVclK+/ogsnCE/ZhnF2Z0rU=
2022-10-21T05:03:48.581 INFO:teuthology.kill:  smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlLtlODuDcn+YaqOuBh+ysFhl5OnI0LsbrtVnFy09fvAQZ8uidyrQ6javstWee0y4EXCwFDbIyyn/sZduiXEdhJarNzwv1S4CiIaZ+kOFAItE9Pum0TQjW+CzeIx7AFPc1j5NJQ7CgjfG04aD/zxezaiv+huy3oVG5T8a51ZnG6lQFnQcrwy81GSukGMtUdhm7YZApWF9vrvhxRuqtzM5DJBCIaBkmkLkm8CGnLfVrc8f7TKIFbD+6v60Bkp1U7nZLshmJ05+EVfo/2+ZNLNP984nf2IZObecrOgf+w+fpfMfww5bcDsg0AY3XwEKheFLv+sQ0fRs0WG0VQo3QQvrnvVvVouhbacJ2g4jx6XiMFj4bM6w/Yljg3g3c4T/5oQXbR+rrHXnCqEbgRxRbHoz2kLC68RtsKyZalAuuv9orIfqsWyIEWlZA2eDtVahO7Olclbs3yZP1WuPQZqTwWKAQ7TedGZukfjtzVbLBhs+z95KRNql49c1gCo1X4PuhM7s=
2022-10-21T05:03:48.581 INFO:teuthology.kill:2022-10-21 05:03:48,579.579 INFO:teuthology.nuke:Checking targets against current locks
2022-10-21T05:03:48.631 INFO:teuthology.kill:2022-10-21 05:03:48,631.631 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-10-21T05:03:48.653 INFO:teuthology.kill:2022-10-21 05:03:48,653.653 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-10-21T05:03:48.711 INFO:teuthology.kill:2022-10-21 05:03:48,711.711 INFO:teuthology.orchestra.console:Power off smithi006
2022-10-21T05:03:48.744 INFO:teuthology.kill:2022-10-21 05:03:48,743.743 INFO:teuthology.orchestra.console:Power off smithi018
2022-10-21T05:04:01.192 INFO:teuthology.kill:2022-10-21 05:04:01,191.191 INFO:teuthology.orchestra.console:Power off for smithi006 completed
2022-10-21T05:04:01.204 INFO:teuthology.kill:2022-10-21 05:04:01,204.204 INFO:teuthology.orchestra.console:Power off for smithi018 completed
2022-10-21T05:04:01.339 INFO:teuthology.kill:2022-10-21 05:04:01,339.339 INFO:teuthology.lock.ops:unlocked: smithi006.front.sepia.ceph.com
2022-10-21T05:04:01.346 INFO:teuthology.kill:2022-10-21 05:04:01,346.346 INFO:teuthology.lock.ops:unlocked: smithi018.front.sepia.ceph.com
2022-10-21T05:06:01.541 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-10-21T05:06:01.657 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi006 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-10-20_14:33:23-fs-wip-yuri2-testing-2022-10-19-1252-pacific-distro-default-smithi/7075736
2022-10-21T05:06:01.657 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi018 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-10-20_14:33:23-fs-wip-yuri2-testing-2022-10-19-1252-pacific-distro-default-smithi/7075736