2022-10-12T18:53:08.781 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-12T18:53:08.916 INFO:teuthology.lock.ops:Start node 'smithi107.front.sepia.ceph.com' reimaging 2022-10-12T18:53:08.917 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: reset os type and version on server 2022-10-12T18:53:08.918 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2022-10-12T18:53:08.938 INFO:teuthology.lock.ops:Node 'smithi107.front.sepia.ceph.com' reimaging is complete 2022-10-12T18:53:08.938 INFO:teuthology.lock.ops:Start node 'smithi182.front.sepia.ceph.com' reimaging 2022-10-12T18:53:08.938 INFO:teuthology.lock.ops:Updating [smithi182.front.sepia.ceph.com]: reset os type and version on server 2022-10-12T18:53:08.939 INFO:teuthology.lock.ops:Updating smithi182.front.sepia.ceph.com on lock server 2022-10-12T18:53:08.957 INFO:teuthology.lock.ops:Node 'smithi182.front.sepia.ceph.com' reimaging is complete 2022-10-12T18:53:09.151 INFO:teuthology.provision.fog.smithi107:Scheduling deploy of centos 8.stream 2022-10-12T18:53:09.161 INFO:teuthology.provision.fog.smithi182:Scheduling deploy of centos 8.stream 2022-10-12T18:53:09.472 INFO:teuthology.orchestra.console:Power off smithi182 2022-10-12T18:53:09.486 INFO:teuthology.orchestra.console:Power off smithi107 2022-10-12T18:53:17.844 INFO:teuthology.orchestra.console:Power off for smithi182 completed 2022-10-12T18:53:17.848 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2022-10-12T18:53:17.946 INFO:teuthology.orchestra.console:Power on smithi182 2022-10-12T18:53:17.954 INFO:teuthology.orchestra.console:Power on smithi107 2022-10-12T18:53:26.340 INFO:teuthology.orchestra.console:Power on for smithi182 completed 2022-10-12T18:53:26.441 INFO:teuthology.provision.fog.smithi182:Waiting for deploy to finish 2022-10-12T18:53:26.463 INFO:teuthology.orchestra.console:Power on for smithi107 completed 2022-10-12T18:53:26.564 INFO:teuthology.provision.fog.smithi107:Waiting for deploy to finish 2022-10-12T18:55:57.566 ERROR:teuthology.orchestra.connection:Error authenticating with smithi107.front.sepia.ceph.com: Authentication failed. 2022-10-12T18:57:38.271 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-12T18:58:03.683 INFO:teuthology.provision.fog.smithi107:Node is ready 2022-10-12T18:58:03.746 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107.front.sepia.ceph.com 2022-10-12T18:58:03.807 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107 smithi107.front.sepia.ceph.com smithi107 2022-10-12T18:58:04.228 INFO:teuthology.provision.fog.smithi107:Deploy complete! 2022-10-12T18:58:04.429 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-12T18:58:04.709 INFO:teuthology.provision.fog.smithi182:Node is ready 2022-10-12T18:58:04.728 INFO:teuthology.orchestra.run.smithi182.stdout:smithi182.front.sepia.ceph.com 2022-10-12T18:58:04.787 INFO:teuthology.orchestra.run.smithi182.stdout:172.21.15.182 smithi182.front.sepia.ceph.com smithi182 2022-10-12T18:58:05.132 INFO:teuthology.provision.fog.smithi182:Deploy complete! 2022-10-12T18:58:05.339 INFO:teuthology.lock.ops:Checking smithi107.front.sepia.ceph.com 2022-10-12T18:58:05.340 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-12T18:58:05.362 INFO:teuthology.lock.ops:Checking smithi182.front.sepia.ceph.com 2022-10-12T18:58:05.363 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-12T18:58:05.382 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: set os type and version on server 2022-10-12T18:58:05.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2022-10-12T18:58:05.588 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2022-10-12T18:58:05.647 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2022-10-12T18:58:05.647 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="8" 2022-10-12T18:58:05.647 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2022-10-12T18:58:05.648 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2022-10-12T18:58:05.648 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="8" 2022-10-12T18:58:05.648 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el8" 2022-10-12T18:58:05.648 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-12T18:58:05.649 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2022-10-12T18:58:05.649 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-12T18:58:05.649 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2022-10-12T18:58:05.649 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-12T18:58:05.650 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-12T18:58:05.650 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-12T18:58:05.651 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2022-10-12T18:58:05.671 INFO:teuthology.lock.ops:Updating [smithi182.front.sepia.ceph.com]: set os type and version on server 2022-10-12T18:58:05.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi182.front.sepia.ceph.com' 2022-10-12T18:58:05.871 INFO:teuthology.orchestra.run.smithi182.stdout:x86_64 2022-10-12T18:58:05.929 INFO:teuthology.orchestra.run.smithi182.stdout:NAME="CentOS Stream" 2022-10-12T18:58:05.929 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION="8" 2022-10-12T18:58:05.930 INFO:teuthology.orchestra.run.smithi182.stdout:ID="centos" 2022-10-12T18:58:05.930 INFO:teuthology.orchestra.run.smithi182.stdout:ID_LIKE="rhel fedora" 2022-10-12T18:58:05.930 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION_ID="8" 2022-10-12T18:58:05.930 INFO:teuthology.orchestra.run.smithi182.stdout:PLATFORM_ID="platform:el8" 2022-10-12T18:58:05.931 INFO:teuthology.orchestra.run.smithi182.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-12T18:58:05.931 INFO:teuthology.orchestra.run.smithi182.stdout:ANSI_COLOR="0;31" 2022-10-12T18:58:05.931 INFO:teuthology.orchestra.run.smithi182.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-12T18:58:05.931 INFO:teuthology.orchestra.run.smithi182.stdout:HOME_URL="https://centos.org/" 2022-10-12T18:58:05.932 INFO:teuthology.orchestra.run.smithi182.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-12T18:58:05.932 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-12T18:58:05.932 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-12T18:58:05.933 INFO:teuthology.lock.ops:Updating smithi182.front.sepia.ceph.com on lock server 2022-10-12T18:58:06.038 INFO:teuthology.dispatcher.supervisor:Running job 7064206 2022-10-12T18:58:06.038 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-10-12_17:33:04-fs-wip-vshankar-testing-20221011-145847-testing-default-smithi/7064206 --name vshankar-2022-10-12_17:33:04-fs-wip-vshankar-testing-20221011-145847-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} -- /home/teuthworker/archive/vshankar-2022-10-12_17:33:04-fs-wip-vshankar-testing-20221011-145847-testing-default-smithi/7064206/orig.config.yaml 2022-10-12T18:58:06.044 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-10-12_17:33:04-fs-wip-vshankar-testing-20221011-145847-testing-default-smithi/7064206 2022-10-12T18:58:06.045 INFO:teuthology.dispatcher.supervisor:Job PID: 22024 2022-10-12T18:58:06.046 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-10-13T06:58:27.968 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-10-13T06:58:28.175 INFO:teuthology.kill:Killing Pids: {22024} 2022-10-13T06:58:29.603 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-10-13T06:58:29.604 INFO:teuthology.task.internal:roles: ubuntu@smithi182.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-10-13T06:58:29.605 INFO:teuthology.misc:Compressing logs... 2022-10-13T06:58:29.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2022-10-13T06:58:29.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi182.front.sepia.ceph.com' 2022-10-13T06:58:30.397 INFO:teuthology.misc:Compressing logs... 2022-10-13T06:58:30.545 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-10-13T06:58:30.768 INFO:teuthology.misc:Compressing logs... 2022-10-13T07:02:47.004 INFO:teuthology.kill:No teuthology processes running 2022-10-13T07:02:48.364 INFO:teuthology.kill:Nuking machines: ['smithi107', 'smithi182'] 2022-10-13T07:02:49.519 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-10-13T07:02:49.520 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-10-13T07:02:49.521 INFO:teuthology.kill:2022-10-13 07:02:49,519.519 INFO:teuthology.nuke:targets: 2022-10-13T07:02:49.521 INFO:teuthology.kill: smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCal/MQm3VjvX/xgUl/40rveyVZe9/oVFDXmahCqOAcWzsivT+CtJGhyDbwGrF1tyg/YjLfMj3Bfx0ranKdeev5tfxNwt+rd2GmqeIseilgu+LQdkSY6cRfCiTDBRo5iR2+xTWHDDQyMCqbrEqwdN+hWWXdzO1ohpo9oNsViiV4rzw8fcijwKhjrlc4gwOEQSuITw/DNkwTqegeCZI3hLPuqfknzeqlIB6AXsEPV16ap7iHBzs5MAa/0Ps55D1QHe4/M8R+0c8kZqrIMX0HHlVKQtHZvHijbwjG0r/nPU2LCvzrgFnADBseV3QnbRWNTmwClioKc0IfFsz/CNoLgNFw8H6O6HOn6OPIRsEZBNc1tM8NgBiHnJvIYxkdU4YLo/Gw3bm7k5EbV9wZPQrZitUzgprd0aAxFxWPBPrMXhvNRCIz3gswLwyDWjMPeEQNVDBc3XyplwY5gGLmBOafyUp1IOgxaoLvuRz2oRqvQD+xWBAWyifA5xDFmeL8lYuD720= 2022-10-13T07:02:49.521 INFO:teuthology.kill: smithi182.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGQqN6TvNgbvD416omaX9y7ivr3lEJLfGhkwgf+IWwP0AEbPTEUchW6kTpxblKFNNjF6+yGJ5qFuycd+HuVYzTeiCOiFs9uAEDJsvX8GDhPNuYopvkYtpJaAbsaBylPeLWzoMIf91uEL8sbB2D8ZSWUFcH59vBR2UStw5lYHDzFmoAlL0XDNgqG1s4QpPWTycrfAclScrqGTtcpW3i65isJLu0hwc8G7sNfTj4xrtPjO/4ybHDiucB6Lk8LRcxTvi8h8C6HnDSBxbRQJACAr+Ggmxo6YhN//eGVmx5cDNkheCEiXBytYBJ0yyC1LrWczAszVyNhb6bPdfrLg7PeG0xNS4irV2dBRrY3cKlVJ1Vw7NZQB78juZkzFlZgWThoUfpmBz8GtMwshJcUotq4CvhTreNszC88M1EEruMbkit4AVOp4jreZ/GiuKVyMt8eddTB08vf5TThb44cYjf/FrdXhntQDmNSDco2yGLDefGbLClcaa6J7LHtv2nGOB2n0s= 2022-10-13T07:02:49.522 INFO:teuthology.kill:2022-10-13 07:02:49,519.519 INFO:teuthology.nuke:Checking targets against current locks 2022-10-13T07:02:49.583 INFO:teuthology.kill:2022-10-13 07:02:49,583.583 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-13T07:02:49.601 INFO:teuthology.kill:2022-10-13 07:02:49,601.601 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-10-13T07:02:49.672 INFO:teuthology.kill:2022-10-13 07:02:49,671.671 INFO:teuthology.orchestra.console:Power off smithi107 2022-10-13T07:02:49.684 INFO:teuthology.kill:2022-10-13 07:02:49,684.684 INFO:teuthology.orchestra.console:Power off smithi182 2022-10-13T07:03:02.156 INFO:teuthology.kill:2022-10-13 07:03:02,155.155 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2022-10-13T07:03:02.158 INFO:teuthology.kill:2022-10-13 07:03:02,158.158 INFO:teuthology.orchestra.console:Power off for smithi182 completed 2022-10-13T07:03:02.314 INFO:teuthology.kill:2022-10-13 07:03:02,313.313 INFO:teuthology.lock.ops:unlocked: smithi107.front.sepia.ceph.com 2022-10-13T07:03:02.378 INFO:teuthology.kill:2022-10-13 07:03:02,378.378 INFO:teuthology.lock.ops:unlocked: smithi182.front.sepia.ceph.com 2022-10-13T07:05:02.616 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-10-13T07:05:02.732 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi107 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-10-13_06:37:57-fs-wip-vshankar-testing-20221013-101155-testing-default-smithi/7064833 2022-10-13T07:05:02.732 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi182 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-10-13_06:37:57-fs-wip-vshankar-testing-20221013-101155-testing-default-smithi/7064833