2021-06-04T21:57:00.808 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-04T21:57:00.900 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi031.front.sepia.ceph.com' reimaging 2021-06-04T21:57:00.901 INFO:teuthology.lock.ops:Updating [ubuntu@smithi031.front.sepia.ceph.com]: reset os type and version on server 2021-06-04T21:57:00.902 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2021-06-04T21:57:00.916 INFO:teuthology.lock.ops:Node 'ubuntu@smithi031.front.sepia.ceph.com' reimaging is complete 2021-06-04T21:57:00.917 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi075.front.sepia.ceph.com' reimaging 2021-06-04T21:57:00.917 INFO:teuthology.lock.ops:Updating [ubuntu@smithi075.front.sepia.ceph.com]: reset os type and version on server 2021-06-04T21:57:00.918 INFO:teuthology.lock.ops:Updating smithi075.front.sepia.ceph.com on lock server 2021-06-04T21:57:00.933 INFO:teuthology.lock.ops:Node 'ubuntu@smithi075.front.sepia.ceph.com' reimaging is complete 2021-06-04T21:57:01.122 INFO:teuthology.provision.fog.smithi031:Scheduling deploy of centos 8.2 2021-06-04T21:57:01.142 INFO:teuthology.provision.fog.smithi075:Scheduling deploy of centos 8.2 2021-06-04T21:57:01.465 INFO:teuthology.orchestra.console:Power off smithi031 2021-06-04T21:57:01.507 INFO:teuthology.orchestra.console:Power off smithi075 2021-06-04T21:57:09.839 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2021-06-04T21:57:09.941 INFO:teuthology.orchestra.console:Power on smithi031 2021-06-04T21:57:13.990 INFO:teuthology.orchestra.console:Power off for smithi075 completed 2021-06-04T21:57:14.092 INFO:teuthology.orchestra.console:Power on smithi075 2021-06-04T21:57:14.191 INFO:teuthology.orchestra.console:Power on for smithi031 completed 2021-06-04T21:57:14.292 INFO:teuthology.provision.fog.smithi031:Waiting for deploy to finish 2021-06-04T21:57:22.459 INFO:teuthology.orchestra.console:Power on for smithi075 completed 2021-06-04T21:57:22.561 INFO:teuthology.provision.fog.smithi075:Waiting for deploy to finish 2021-06-04T22:00:56.514 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-04T22:01:12.399 INFO:teuthology.provision.fog.smithi031:Node is ready 2021-06-04T22:01:12.421 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031.front.sepia.ceph.com 2021-06-04T22:01:12.480 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31 smithi031.front.sepia.ceph.com smithi031 2021-06-04T22:01:12.846 INFO:teuthology.provision.fog.smithi031:Deploy complete! 2021-06-04T22:02:03.468 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-04T22:02:03.771 INFO:teuthology.provision.fog.smithi075:Node is ready 2021-06-04T22:02:03.786 INFO:teuthology.orchestra.run.smithi075.stdout:smithi075.front.sepia.ceph.com 2021-06-04T22:02:03.841 INFO:teuthology.orchestra.run.smithi075.stdout:172.21.15.75 smithi075.front.sepia.ceph.com smithi075 2021-06-04T22:02:04.158 INFO:teuthology.provision.fog.smithi075:Deploy complete! 2021-06-04T22:02:04.317 INFO:teuthology.lock.ops:Checking smithi031.front.sepia.ceph.com 2021-06-04T22:02:04.318 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-04T22:02:04.337 INFO:teuthology.lock.ops:Checking smithi075.front.sepia.ceph.com 2021-06-04T22:02:04.337 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-04T22:02:04.354 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: set os type and version on server 2021-06-04T22:02:04.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-04T22:02:04.641 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2021-06-04T22:02:04.697 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Linux" 2021-06-04T22:02:04.697 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8 (Core)" 2021-06-04T22:02:04.698 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2021-06-04T22:02:04.698 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2021-06-04T22:02:04.699 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8" 2021-06-04T22:02:04.699 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2021-06-04T22:02:04.700 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-04T22:02:04.700 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2021-06-04T22:02:04.700 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-04T22:02:04.701 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.centos.org/" 2021-06-04T22:02:04.701 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-04T22:02:04.702 INFO:teuthology.orchestra.run.smithi031.stdout: 2021-06-04T22:02:04.702 INFO:teuthology.orchestra.run.smithi031.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-04T22:02:04.702 INFO:teuthology.orchestra.run.smithi031.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-04T22:02:04.703 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-04T22:02:04.703 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-04T22:02:04.704 INFO:teuthology.orchestra.run.smithi031.stdout: 2021-06-04T22:02:04.706 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2021-06-04T22:02:04.724 INFO:teuthology.lock.ops:Updating [smithi075.front.sepia.ceph.com]: set os type and version on server 2021-06-04T22:02:04.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-04T22:02:04.999 INFO:teuthology.orchestra.run.smithi075.stdout:x86_64 2021-06-04T22:02:05.053 INFO:teuthology.orchestra.run.smithi075.stdout:NAME="CentOS Linux" 2021-06-04T22:02:05.054 INFO:teuthology.orchestra.run.smithi075.stdout:VERSION="8 (Core)" 2021-06-04T22:02:05.054 INFO:teuthology.orchestra.run.smithi075.stdout:ID="centos" 2021-06-04T22:02:05.054 INFO:teuthology.orchestra.run.smithi075.stdout:ID_LIKE="rhel fedora" 2021-06-04T22:02:05.054 INFO:teuthology.orchestra.run.smithi075.stdout:VERSION_ID="8" 2021-06-04T22:02:05.055 INFO:teuthology.orchestra.run.smithi075.stdout:PLATFORM_ID="platform:el8" 2021-06-04T22:02:05.055 INFO:teuthology.orchestra.run.smithi075.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-04T22:02:05.055 INFO:teuthology.orchestra.run.smithi075.stdout:ANSI_COLOR="0;31" 2021-06-04T22:02:05.055 INFO:teuthology.orchestra.run.smithi075.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-04T22:02:05.055 INFO:teuthology.orchestra.run.smithi075.stdout:HOME_URL="https://www.centos.org/" 2021-06-04T22:02:05.056 INFO:teuthology.orchestra.run.smithi075.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-04T22:02:05.056 INFO:teuthology.orchestra.run.smithi075.stdout: 2021-06-04T22:02:05.056 INFO:teuthology.orchestra.run.smithi075.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-04T22:02:05.056 INFO:teuthology.orchestra.run.smithi075.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-04T22:02:05.056 INFO:teuthology.orchestra.run.smithi075.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-04T22:02:05.057 INFO:teuthology.orchestra.run.smithi075.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-04T22:02:05.057 INFO:teuthology.orchestra.run.smithi075.stdout: 2021-06-04T22:02:05.058 INFO:teuthology.lock.ops:Updating smithi075.front.sepia.ceph.com on lock server 2021-06-04T22:02:05.148 INFO:teuthology.dispatcher.supervisor:Running job 6152546 2021-06-04T22:02:05.148 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-06-04_18:17:44-fs-wip-yuri4-testing-2021-06-04-0748-pacific-distro-basic-smithi/6152546 --name yuriw-2021-06-04_18:17:44-fs-wip-yuri4-testing-2021-06-04-0748-pacific-distro-basic-smithi --description fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} -- /home/teuthworker/archive/yuriw-2021-06-04_18:17:44-fs-wip-yuri4-testing-2021-06-04-0748-pacific-distro-basic-smithi/6152546/orig.config.yaml 2021-06-04T22:02:05.152 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-06-04_18:17:44-fs-wip-yuri4-testing-2021-06-04-0748-pacific-distro-basic-smithi/6152546 2021-06-04T22:02:05.153 INFO:teuthology.dispatcher.supervisor:Job PID: 10447 2021-06-04T22:02:05.154 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-05T10:02:54.875 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-05T10:02:55.084 INFO:teuthology.kill:Killing Pids: {10447} 2021-06-05T10:02:56.234 INFO:teuthology.kill:Nuking machines: ['smithi031', 'smithi075'] 2021-06-05T10:02:57.165 INFO:teuthology.kill:2021-06-05 10:02:57,162.162 INFO:teuthology.nuke:targets: 2021-06-05T10:02:57.166 INFO:teuthology.kill: smithi031.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzUJOkSuFh6OaDJUuG6/u8rtj4dWKPK9Gh+yzBa/wNa3PEkmC/VkZlbkeICC0OrS0DQ9UbYuUTMDpwamdVzZoPT7KNWqQeftmtgNN+GK5pK1oXaFELYImqq0iBfMeWnaoyv6RVrFN+FP+IUIu2ACIcX8BwPJMZ72svMEja8MLXLY8CRwrXiMJl0zpdsG5A3BTXzN41LoeZv6VBpsHHHc+mPJqqFHdS8ElrT22iICi0u1xXwNzAEgNDHE+IZ2IYpAyHEJDCFPUxkjVcwAyedvZJx1k7MvOhUfFpyfYalIhIpf+qUZ9Wr0n/gg+78mfcbNplRn31mB0rkrwmEkYbSu7M20XxHi3DUqOySVCm9dKHSVuAkYyMJ26V49mmXi49/qehmJ7ClD2ShnvoUqS/E3DoU2mH6kHQ8rNm0SnFPyErTjjWrvf1tUQJlTJkZOkq+GocwXwC1Btvog0f2rCORwKqt7lyQqh7kHExAVSpUAtcZerPp3eP2sbsqn34/7Vj3xs= 2021-06-05T10:02:57.166 INFO:teuthology.kill: smithi075.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe6Y8toenD/z05gnoR7xUQtlhg9jNEigzO4y2EuPobNomnxbz2difn+2g8WNPSH6Vjhbw7BHVJaBc1ZpLmh5htiPMwe9MrW8gjcA/go+ik9Ym60K6irnxuvHomNdL6fNoTVdXCXzpfv0bAGw7luS76D+xAVPrE0ah0UZaMcg/bFkZ/MhYDm5Ly1VyZ4avfGYFcWbtRXJqZ9+iyJrollfBP4VTG9ofRXI1DyS/KJwQc8jpOA9U8BNE8qtf6WLOmnoqSwAQWRlHr3KABQWsgYE4yh7T9+sn4hgbDD/I4nttKJWjxJ9MsqF8w1DnQzUj3RsG02RVHr6+1bgdYhrpXskhEUky9GFVW/konmZarZjy3aMqIW9OoRULn0ZevPtmxVjhYJr3G573uJMwNXQVGFcnWrwvYboSkSLXpqPgkYyQXrXR9fqXe7F2+PqCYNysObwlJ4srOkkbd3PNXP+j7M3jDZkUa4Uu7Qnr33vSpslwvOUFqd4ugtUhkWWs6tZYv9Rs= 2021-06-05T10:02:57.166 INFO:teuthology.kill:2021-06-05 10:02:57,162.162 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-05T10:02:57.167 INFO:teuthology.kill:2021-06-05 10:02:57,165.165 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-05T10:02:57.226 INFO:teuthology.kill:2021-06-05 10:02:57,225.225 INFO:teuthology.orchestra.console:Power off smithi031 2021-06-05T10:02:57.239 INFO:teuthology.kill:2021-06-05 10:02:57,239.239 INFO:teuthology.orchestra.console:Power off smithi075 2021-06-05T10:03:05.601 INFO:teuthology.kill:2021-06-05 10:03:05,600.600 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2021-06-05T10:03:09.715 INFO:teuthology.kill:2021-06-05 10:03:09,715.715 INFO:teuthology.orchestra.console:Power off for smithi075 completed 2021-06-05T10:03:09.953 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-06-05T10:03:09.953 INFO:teuthology.task.internal:roles: ubuntu@smithi075.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2021-06-05T10:03:09.954 INFO:teuthology.misc:Compressing logs... 2021-06-05T10:03:09.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-05T10:04:10.019 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-05T10:04:10.093 INFO:teuthology.kill:No teuthology processes running 2021-06-05T10:04:11.137 INFO:teuthology.kill:Nuking machines: ['smithi031', 'smithi075'] 2021-06-05T10:04:12.097 INFO:teuthology.kill:2021-06-05 10:04:12,097.097 INFO:teuthology.nuke:targets: 2021-06-05T10:04:12.098 INFO:teuthology.kill: smithi031.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzUJOkSuFh6OaDJUuG6/u8rtj4dWKPK9Gh+yzBa/wNa3PEkmC/VkZlbkeICC0OrS0DQ9UbYuUTMDpwamdVzZoPT7KNWqQeftmtgNN+GK5pK1oXaFELYImqq0iBfMeWnaoyv6RVrFN+FP+IUIu2ACIcX8BwPJMZ72svMEja8MLXLY8CRwrXiMJl0zpdsG5A3BTXzN41LoeZv6VBpsHHHc+mPJqqFHdS8ElrT22iICi0u1xXwNzAEgNDHE+IZ2IYpAyHEJDCFPUxkjVcwAyedvZJx1k7MvOhUfFpyfYalIhIpf+qUZ9Wr0n/gg+78mfcbNplRn31mB0rkrwmEkYbSu7M20XxHi3DUqOySVCm9dKHSVuAkYyMJ26V49mmXi49/qehmJ7ClD2ShnvoUqS/E3DoU2mH6kHQ8rNm0SnFPyErTjjWrvf1tUQJlTJkZOkq+GocwXwC1Btvog0f2rCORwKqt7lyQqh7kHExAVSpUAtcZerPp3eP2sbsqn34/7Vj3xs= 2021-06-05T10:04:12.098 INFO:teuthology.kill: smithi075.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe6Y8toenD/z05gnoR7xUQtlhg9jNEigzO4y2EuPobNomnxbz2difn+2g8WNPSH6Vjhbw7BHVJaBc1ZpLmh5htiPMwe9MrW8gjcA/go+ik9Ym60K6irnxuvHomNdL6fNoTVdXCXzpfv0bAGw7luS76D+xAVPrE0ah0UZaMcg/bFkZ/MhYDm5Ly1VyZ4avfGYFcWbtRXJqZ9+iyJrollfBP4VTG9ofRXI1DyS/KJwQc8jpOA9U8BNE8qtf6WLOmnoqSwAQWRlHr3KABQWsgYE4yh7T9+sn4hgbDD/I4nttKJWjxJ9MsqF8w1DnQzUj3RsG02RVHr6+1bgdYhrpXskhEUky9GFVW/konmZarZjy3aMqIW9OoRULn0ZevPtmxVjhYJr3G573uJMwNXQVGFcnWrwvYboSkSLXpqPgkYyQXrXR9fqXe7F2+PqCYNysObwlJ4srOkkbd3PNXP+j7M3jDZkUa4Uu7Qnr33vSpslwvOUFqd4ugtUhkWWs6tZYv9Rs= 2021-06-05T10:04:12.117 INFO:teuthology.kill:2021-06-05 10:04:12,117.117 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-05T10:04:12.120 INFO:teuthology.kill:2021-06-05 10:04:12,120.120 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-05T10:04:12.175 INFO:teuthology.kill:2021-06-05 10:04:12,175.175 INFO:teuthology.orchestra.console:Power off smithi075 2021-06-05T10:04:12.188 INFO:teuthology.kill:2021-06-05 10:04:12,188.188 INFO:teuthology.orchestra.console:Power off smithi031 2021-06-05T10:04:12.318 INFO:teuthology.kill:2021-06-05 10:04:12,318.318 INFO:teuthology.orchestra.console:Power off for smithi075 completed 2021-06-05T10:04:12.326 INFO:teuthology.kill:2021-06-05 10:04:12,326.326 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2021-06-05T10:04:12.462 INFO:teuthology.kill:2021-06-05 10:04:12,461.461 INFO:teuthology.lock.ops:unlocked smithi075.front.sepia.ceph.com 2021-06-05T10:04:12.465 INFO:teuthology.kill:2021-06-05 10:04:12,465.465 INFO:teuthology.lock.ops:unlocked smithi031.front.sepia.ceph.com 2021-06-05T10:06:12.770 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-05T10:06:12.865 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-05T10:06:12.866 INFO:teuthology.nuke:Checking targets against current locks 2021-06-05T10:06:12.933 INFO:teuthology.nuke:Not nuking smithi075.front.sepia.ceph.com because description doesn't match 2021-06-05T10:06:12.933 INFO:teuthology.nuke:Not nuking smithi031.front.sepia.ceph.com because description doesn't match