2021-04-03T20:47:09.109 INFO:root:teuthology version: 1.1.0-85d61eae 2021-04-03T20:47:09.251 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging 2021-04-03T20:47:09.252 INFO:teuthology.lock.ops:Updating [ubuntu@smithi002.front.sepia.ceph.com]: reset os type and version on server 2021-04-03T20:47:09.253 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2021-04-03T20:47:09.269 INFO:teuthology.lock.ops:Node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging is complete 2021-04-03T20:47:09.270 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi107.front.sepia.ceph.com' reimaging 2021-04-03T20:47:09.270 INFO:teuthology.lock.ops:Updating [ubuntu@smithi107.front.sepia.ceph.com]: reset os type and version on server 2021-04-03T20:47:09.270 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2021-04-03T20:47:09.289 INFO:teuthology.lock.ops:Node 'ubuntu@smithi107.front.sepia.ceph.com' reimaging is complete 2021-04-03T20:47:09.494 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of rhel 8.2 2021-04-03T20:47:09.542 INFO:teuthology.provision.fog.smithi107:Scheduling deploy of rhel 8.2 2021-04-03T20:47:10.115 INFO:teuthology.orchestra.console:Power off smithi002 2021-04-03T20:47:10.150 INFO:teuthology.orchestra.console:Power off smithi107 2021-04-03T20:47:18.480 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-04-03T20:47:18.510 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2021-04-03T20:47:18.582 INFO:teuthology.orchestra.console:Power on smithi002 2021-04-03T20:47:18.612 INFO:teuthology.orchestra.console:Power on smithi107 2021-04-03T20:47:26.945 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2021-04-03T20:47:26.970 INFO:teuthology.orchestra.console:Power on for smithi107 completed 2021-04-03T20:47:27.047 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2021-04-03T20:47:27.071 INFO:teuthology.provision.fog.smithi107:Waiting for deploy to finish 2021-04-03T20:51:34.906 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-03T20:51:35.242 INFO:teuthology.provision.fog.smithi002:Node is ready 2021-04-03T20:51:35.257 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2021-04-03T20:51:35.313 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2021-04-03T20:51:35.647 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2021-04-03T20:52:08.116 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-03T20:52:08.452 INFO:teuthology.provision.fog.smithi107:Node is ready 2021-04-03T20:52:08.469 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107.front.sepia.ceph.com 2021-04-03T20:52:08.525 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107 smithi107.front.sepia.ceph.com smithi107 2021-04-03T20:52:08.857 INFO:teuthology.provision.fog.smithi107:Deploy complete! 2021-04-03T20:52:08.987 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2021-04-03T20:52:08.990 INFO:teuthology.lock.ops:New key found. Updating... 2021-04-03T20:52:09.010 INFO:teuthology.lock.ops:Checking smithi107.front.sepia.ceph.com 2021-04-03T20:52:09.011 INFO:teuthology.lock.ops:New key found. Updating... 2021-04-03T20:52:09.028 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2021-04-03T20:52:09.028 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T20:52:09.337 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2021-04-03T20:52:09.392 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Red Hat Enterprise Linux" 2021-04-03T20:52:09.392 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8.2 (Ootpa)" 2021-04-03T20:52:09.392 INFO:teuthology.orchestra.run.smithi002.stdout:ID="rhel" 2021-04-03T20:52:09.393 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="fedora" 2021-04-03T20:52:09.393 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8.2" 2021-04-03T20:52:09.393 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2021-04-03T20:52:09.393 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.2 (Ootpa)" 2021-04-03T20:52:09.393 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2021-04-03T20:52:09.394 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.2:GA" 2021-04-03T20:52:09.394 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.redhat.com/" 2021-04-03T20:52:09.394 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-04-03T20:52:09.394 INFO:teuthology.orchestra.run.smithi002.stdout: 2021-04-03T20:52:09.395 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-04-03T20:52:09.395 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.2 2021-04-03T20:52:09.395 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-04-03T20:52:09.395 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.2" 2021-04-03T20:52:09.397 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2021-04-03T20:52:09.414 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: set os type and version on server 2021-04-03T20:52:09.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-03T20:52:09.695 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2021-04-03T20:52:09.749 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Red Hat Enterprise Linux" 2021-04-03T20:52:09.750 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="8.2 (Ootpa)" 2021-04-03T20:52:09.750 INFO:teuthology.orchestra.run.smithi107.stdout:ID="rhel" 2021-04-03T20:52:09.750 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="fedora" 2021-04-03T20:52:09.751 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="8.2" 2021-04-03T20:52:09.751 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el8" 2021-04-03T20:52:09.752 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.2 (Ootpa)" 2021-04-03T20:52:09.752 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2021-04-03T20:52:09.753 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.2:GA" 2021-04-03T20:52:09.753 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.redhat.com/" 2021-04-03T20:52:09.753 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-04-03T20:52:09.754 INFO:teuthology.orchestra.run.smithi107.stdout: 2021-04-03T20:52:09.754 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-04-03T20:52:09.755 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.2 2021-04-03T20:52:09.755 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-04-03T20:52:09.756 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.2" 2021-04-03T20:52:09.756 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2021-04-03T20:52:09.844 INFO:teuthology.dispatcher.supervisor:Running job 6018441 2021-04-03T20:52:09.845 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-04-03_17:21:06-rados-wip-yuri2-testing-2021-04-01-1440-octopus-distro-basic-smithi/6018441 --name yuriw-2021-04-03_17:21:06-rados-wip-yuri2-testing-2021-04-01-1440-octopus-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-small} -- /home/teuthworker/archive/yuriw-2021-04-03_17:21:06-rados-wip-yuri2-testing-2021-04-01-1440-octopus-distro-basic-smithi/6018441/orig.config.yaml 2021-04-03T20:52:09.851 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-04-03_17:21:06-rados-wip-yuri2-testing-2021-04-01-1440-octopus-distro-basic-smithi/6018441 2021-04-03T20:52:09.852 INFO:teuthology.dispatcher.supervisor:Job PID: 29623 2021-04-03T20:52:09.852 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-04-04T08:52:20.696 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-04-04T08:52:20.971 INFO:teuthology.kill:Killing Pids: {29623} 2021-04-04T08:52:22.552 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi107'] 2021-04-04T08:52:23.849 INFO:teuthology.kill:2021-04-04 08:52:23,848.848 INFO:teuthology.nuke:targets: 2021-04-04T08:52:23.850 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDy9BK9IXjL0hDfyHdMRvhribd/MUuK8SIiSdlWsgYh8nGBwPewQMs/V6aGPKChGtZvEUn8JKUuvd7ITjTkS7BcNq4I69+j1RSZjWH7FSY7WrL4cA1JuZwxWJa4OAM+ETcbueMNrJFGhcxzAKSeqmpFE4cHUbdeSUU2O1Kq3w/nuHJ/Hyqdph0QUSrzsJviEcu4W87GLfKcKLoI4fdEAAW87Fma7ze7OA90MbOatP5/Ta0So/RbvOQa4PHvYkGsXNwSSlVuDFd+ZnTruZUGB+kp/riUGpAqh6IigcEAE3dnYlgu/bgE/N5EijuJyrB0IjNIDGNHGBa0a0XCnXNeoUclyveKpEJu3ZPmUhstKTyd24eIimgeF+Soz6thxgPOLK2uHd6YLfxlWe99mI2TM0imex5NHVnS5UwOjzqnnXmqvfc4LMA/QCasj/LAGXyo/H80lQ6HHIxLWkFQthzoZPRzR1raRrFY4vWqMcU954Z/U7a+EayLShG1lQUE5cl1RXs= 2021-04-04T08:52:23.851 INFO:teuthology.kill: smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJySnCOqvIo8NHty4NC4lxhrzOCrQcghDTqGFbcRi+/K2GcvJE4S4+ELz7aiFZ7bo0qYoGqkkZbQpUk12go7U3kiCUN3ylGeqJsQ9x7LBhE18L7idT1pvzyYFnaklSgm+/NiI11EDSRcKbEjkc1fjdfn0GZGfvou4MuXgErX02Mv8qDbP6LmalMayR8j5V8rrARBClINMUsMNshOedutGxapwdTbtZCUY+4Dgk+31D1VcXRgurx2By0hza9cKhiPKcyUC9xOXagHyQfMxjQjcipKs6bRsVdZtBAVtxuPvVCA1/YHevAr0YiM6mm7GfYahwroYEBTflWgnWb4Nps7lSC4HCuqWgyiWkbRwTl/GeOxuHcmwr/Lr36GK3+r+qypZddUYeeKTfL0uD3w5G8iLv0bowP1sOTl8q/fLiAahoempR35XUnBMEIdw00waqG70/DpFeKPi8dGqJ/1x6ipbuiarUXIswQXuCYj4JhtZZ0oDVsN8KeYawQE68y/oh5xM= 2021-04-04T08:52:23.851 INFO:teuthology.kill:2021-04-04 08:52:23,849.849 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-04T08:52:23.852 INFO:teuthology.kill:2021-04-04 08:52:23,851.851 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-04T08:52:23.939 INFO:teuthology.kill:2021-04-04 08:52:23,939.939 INFO:teuthology.orchestra.console:Power off smithi002 2021-04-04T08:52:23.999 INFO:teuthology.kill:2021-04-04 08:52:23,999.999 INFO:teuthology.orchestra.console:Power off smithi107 2021-04-04T08:52:32.424 INFO:teuthology.kill:2021-04-04 08:52:32,424.424 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-04-04T08:52:32.478 INFO:teuthology.kill:2021-04-04 08:52:32,450.450 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2021-04-04T08:52:32.849 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-04-04T08:52:32.859 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-04-04T08:52:32.860 INFO:teuthology.misc:Compressing logs... 2021-04-04T08:52:32.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-04T08:53:32.928 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-04-04T08:53:33.014 INFO:teuthology.kill:No teuthology processes running 2021-04-04T08:53:34.717 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi107'] 2021-04-04T08:53:35.826 INFO:teuthology.kill:2021-04-04 08:53:35,826.826 INFO:teuthology.nuke:targets: 2021-04-04T08:53:35.828 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDy9BK9IXjL0hDfyHdMRvhribd/MUuK8SIiSdlWsgYh8nGBwPewQMs/V6aGPKChGtZvEUn8JKUuvd7ITjTkS7BcNq4I69+j1RSZjWH7FSY7WrL4cA1JuZwxWJa4OAM+ETcbueMNrJFGhcxzAKSeqmpFE4cHUbdeSUU2O1Kq3w/nuHJ/Hyqdph0QUSrzsJviEcu4W87GLfKcKLoI4fdEAAW87Fma7ze7OA90MbOatP5/Ta0So/RbvOQa4PHvYkGsXNwSSlVuDFd+ZnTruZUGB+kp/riUGpAqh6IigcEAE3dnYlgu/bgE/N5EijuJyrB0IjNIDGNHGBa0a0XCnXNeoUclyveKpEJu3ZPmUhstKTyd24eIimgeF+Soz6thxgPOLK2uHd6YLfxlWe99mI2TM0imex5NHVnS5UwOjzqnnXmqvfc4LMA/QCasj/LAGXyo/H80lQ6HHIxLWkFQthzoZPRzR1raRrFY4vWqMcU954Z/U7a+EayLShG1lQUE5cl1RXs= 2021-04-04T08:53:35.829 INFO:teuthology.kill: smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJySnCOqvIo8NHty4NC4lxhrzOCrQcghDTqGFbcRi+/K2GcvJE4S4+ELz7aiFZ7bo0qYoGqkkZbQpUk12go7U3kiCUN3ylGeqJsQ9x7LBhE18L7idT1pvzyYFnaklSgm+/NiI11EDSRcKbEjkc1fjdfn0GZGfvou4MuXgErX02Mv8qDbP6LmalMayR8j5V8rrARBClINMUsMNshOedutGxapwdTbtZCUY+4Dgk+31D1VcXRgurx2By0hza9cKhiPKcyUC9xOXagHyQfMxjQjcipKs6bRsVdZtBAVtxuPvVCA1/YHevAr0YiM6mm7GfYahwroYEBTflWgnWb4Nps7lSC4HCuqWgyiWkbRwTl/GeOxuHcmwr/Lr36GK3+r+qypZddUYeeKTfL0uD3w5G8iLv0bowP1sOTl8q/fLiAahoempR35XUnBMEIdw00waqG70/DpFeKPi8dGqJ/1x6ipbuiarUXIswQXuCYj4JhtZZ0oDVsN8KeYawQE68y/oh5xM= 2021-04-04T08:53:35.858 INFO:teuthology.kill:2021-04-04 08:53:35,857.857 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-04T08:53:35.865 INFO:teuthology.kill:2021-04-04 08:53:35,865.865 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-04T08:53:35.953 INFO:teuthology.kill:2021-04-04 08:53:35,952.952 INFO:teuthology.orchestra.console:Power off smithi002 2021-04-04T08:53:35.976 INFO:teuthology.kill:2021-04-04 08:53:35,975.975 INFO:teuthology.orchestra.console:Power off smithi107 2021-04-04T08:53:36.255 INFO:teuthology.kill:2021-04-04 08:53:36,255.255 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2021-04-04T08:53:36.257 INFO:teuthology.kill:2021-04-04 08:53:36,257.257 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-04-04T08:53:36.410 INFO:teuthology.kill:2021-04-04 08:53:36,409.409 INFO:teuthology.lock.ops:unlocked smithi107.front.sepia.ceph.com 2021-04-04T08:53:36.429 INFO:teuthology.kill:2021-04-04 08:53:36,428.428 INFO:teuthology.lock.ops:unlocked smithi002.front.sepia.ceph.com 2021-04-04T08:55:52.403 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-04-04T08:55:52.583 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-04-04T08:55:52.584 INFO:teuthology.nuke:Checking targets against current locks 2021-04-04T08:55:52.683 INFO:teuthology.nuke:Not nuking smithi107.front.sepia.ceph.com because description doesn't match 2021-04-04T08:55:52.684 INFO:teuthology.nuke:Not nuking smithi002.front.sepia.ceph.com because description doesn't match