2022-07-03T10:49:16.387 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-07-03T10:49:16.544 INFO:teuthology.lock.ops:Start node 'smithi155.front.sepia.ceph.com' reimaging 2022-07-03T10:49:16.545 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: reset os type and version on server 2022-07-03T10:49:16.545 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2022-07-03T10:49:16.566 INFO:teuthology.lock.ops:Node 'smithi155.front.sepia.ceph.com' reimaging is complete 2022-07-03T10:49:16.566 INFO:teuthology.lock.ops:Start node 'smithi185.front.sepia.ceph.com' reimaging 2022-07-03T10:49:16.567 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: reset os type and version on server 2022-07-03T10:49:16.567 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-07-03T10:49:16.588 INFO:teuthology.lock.ops:Node 'smithi185.front.sepia.ceph.com' reimaging is complete 2022-07-03T10:49:16.788 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of centos 8.stream 2022-07-03T10:49:16.814 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of centos 8.stream 2022-07-03T10:49:17.157 INFO:teuthology.orchestra.console:Power off smithi155 2022-07-03T10:49:17.171 INFO:teuthology.orchestra.console:Power off smithi185 2022-07-03T10:49:25.524 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2022-07-03T10:49:25.625 INFO:teuthology.orchestra.console:Power on smithi155 2022-07-03T10:49:29.643 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-07-03T10:49:29.745 INFO:teuthology.orchestra.console:Power on smithi185 2022-07-03T10:49:34.035 INFO:teuthology.orchestra.console:Power on for smithi155 completed 2022-07-03T10:49:34.136 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2022-07-03T10:49:38.108 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2022-07-03T10:49:38.210 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2022-07-03T10:53:55.630 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-03T10:54:21.055 INFO:teuthology.provision.fog.smithi155:Node is ready 2022-07-03T10:54:21.076 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2022-07-03T10:54:21.135 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2022-07-03T10:54:21.516 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2022-07-03T10:54:25.053 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-03T10:54:25.365 INFO:teuthology.provision.fog.smithi185:Node is ready 2022-07-03T10:54:25.385 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2022-07-03T10:54:25.443 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2022-07-03T10:54:25.820 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2022-07-03T10:54:26.011 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2022-07-03T10:54:26.012 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-03T10:54:26.041 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2022-07-03T10:54:26.042 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-03T10:54:26.064 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2022-07-03T10:54:26.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2022-07-03T10:54:26.274 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2022-07-03T10:54:26.333 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2022-07-03T10:54:26.333 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8" 2022-07-03T10:54:26.333 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2022-07-03T10:54:26.334 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2022-07-03T10:54:26.334 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8" 2022-07-03T10:54:26.334 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2022-07-03T10:54:26.334 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-03T10:54:26.335 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2022-07-03T10:54:26.335 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-03T10:54:26.335 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2022-07-03T10:54:26.335 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-03T10:54:26.336 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-03T10:54:26.336 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-03T10:54:26.338 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2022-07-03T10:54:26.360 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2022-07-03T10:54:26.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-07-03T10:54:26.585 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2022-07-03T10:54:26.641 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="CentOS Stream" 2022-07-03T10:54:26.642 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8" 2022-07-03T10:54:26.642 INFO:teuthology.orchestra.run.smithi185.stdout:ID="centos" 2022-07-03T10:54:26.642 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="rhel fedora" 2022-07-03T10:54:26.643 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8" 2022-07-03T10:54:26.643 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8" 2022-07-03T10:54:26.643 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-03T10:54:26.643 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31" 2022-07-03T10:54:26.644 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-03T10:54:26.644 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://centos.org/" 2022-07-03T10:54:26.644 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-03T10:54:26.644 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-03T10:54:26.645 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-03T10:54:26.645 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-07-03T10:54:26.773 INFO:teuthology.dispatcher.supervisor:Running job 6910875 2022-07-03T10:54:26.773 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910875 --name rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} -- /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910875/orig.config.yaml 2022-07-03T10:54:26.778 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910875 2022-07-03T10:54:26.778 INFO:teuthology.dispatcher.supervisor:Job PID: 12113 2022-07-03T10:54:26.779 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-03T22:55:24.415 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-03T22:55:24.582 INFO:teuthology.kill:Killing Pids: {12113} 2022-07-03T22:55:25.985 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-07-03T22:55:25.986 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-07-03T22:55:25.986 INFO:teuthology.misc:Compressing logs... 2022-07-03T22:55:25.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2022-07-03T22:55:27.738 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 290, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 348, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/remote.py", line 509, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi155 2022-07-03T22:55:27.807 INFO:teuthology.kill:No teuthology processes running 2022-07-03T22:55:29.128 INFO:teuthology.kill:Nuking machines: ['smithi155', 'smithi185'] 2022-07-03T22:55:30.278 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-03T22:55:30.279 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-03T22:55:30.279 INFO:teuthology.kill:2022-07-03 22:55:30,277.277 INFO:teuthology.nuke:targets: 2022-07-03T22:55:30.279 INFO:teuthology.kill: smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw3DG3XgG8fyAb5QLkltQRtmk713LN7R4edFJ/CSuH+wPJ1GdhIzOHO0ZG0rzrZT6WCJenWu+0YF/HWdkcZlye3WHcIOOoOp0KOUueb8mTH5OOVpLY0rEnm51hCPi33QyS/RBW4TLuOTlzEiyV6qV4IoHbl9mVXNNqrihtk/eQL7LYPzvzpTD7IIcrYFj4ssD4tYH1nkzQKKDbHkIN1HhqDkIfxCjj1SZ2eZfwM2/U196BA7SHVRmrLkCcoTJSPP39LznEZzHeEJDZTaLde5NnckxHQYhfpJdJ3GIa6I/a2GR7+FUFy3U/W4dlZE/6WccnAbz8SzkWtpLIDqAFqQn1yiXT7+H/wDgjfHxP89hLwhgCO6Qn0VJZA7crhCMvvM967qkcYtvvAsC7j00M59rTWGfbIrEj4pG6+h1IWfPo8I9lRZ9t8iRBmDhyjgTh2TEOIpnI7BxKjylS0lNK4w0/0t8QlSJhLWM8RRDwzb2r1Y0wVT/5sLQS9Z89jVGn0Jk= 2022-07-03T22:55:30.279 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmfLFGAYZH7mLp7DjsaD7ztwX8RRmbfSegXRsITIImE/uPIyUkDK5RhAjuuG9oZm2mLjp2bugRYOaRBSgia3Pw777UB+rrSxWwYO0Jfl3L2EGFakGZuzSTW2WRYhrUC21tIWYETBJPTXx5qFKEMBe+xm0e5hp4YBDw1u3M9NmnayfFIrj5VvoM+2UerEQppqk+2BXql4KSCxIIlqgYyoe0FCsFX+wn/PfgEof6a7RS8TPIawUvLEyLW1k29XkVE1G54U2wu7++mkJio7eT4iRwRc82zqDMrqsuAcxLYcSd3ALDvbx3u4GbhV4+5fUHlEGPCy1jGGQelJ1DYc5NVU2sxdjd0DxxFytHJekPtK0bSMrkqypmda4//dTMX0flZeHRKL0tRi/BQz+ZmJpwWNlmCzklRB5doKtz+YWlqDMBySEGWWDw5fCJHAARg28oMrmc8D2yg+ZgsBq9zTtJli+PZ+ftgB7imsuwBDH7moa2AK5d6hbazs9LInM4YsFrhVs= 2022-07-03T22:55:30.280 INFO:teuthology.kill:2022-07-03 22:55:30,277.277 INFO:teuthology.nuke:Checking targets against current locks 2022-07-03T22:55:30.328 INFO:teuthology.kill:2022-07-03 22:55:30,328.328 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-03T22:55:30.358 INFO:teuthology.kill:2022-07-03 22:55:30,357.357 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-03T22:55:30.412 INFO:teuthology.kill:2022-07-03 22:55:30,412.412 INFO:teuthology.orchestra.console:Power off smithi155 2022-07-03T22:55:30.453 INFO:teuthology.kill:2022-07-03 22:55:30,453.453 INFO:teuthology.orchestra.console:Power off smithi185 2022-07-03T22:55:38.817 INFO:teuthology.kill:2022-07-03 22:55:38,816.816 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-07-03T22:55:38.969 INFO:teuthology.kill:2022-07-03 22:55:38,968.968 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com 2022-07-03T22:55:42.903 INFO:teuthology.kill:2022-07-03 22:55:42,903.903 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2022-07-03T22:55:43.056 INFO:teuthology.kill:2022-07-03 22:55:43,055.055 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com 2022-07-03T22:57:43.381 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-03T22:57:43.499 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi155 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-07-02_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6910939 2022-07-03T22:57:43.500 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-07-02_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6910939