2021-07-02T13:45:49.488 INFO:root:teuthology version: 1.1.0-26631ede 2021-07-02T13:45:49.660 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging 2021-07-02T13:45:49.661 INFO:teuthology.lock.ops:Updating [ubuntu@smithi112.front.sepia.ceph.com]: reset os type and version on server 2021-07-02T13:45:49.661 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2021-07-02T13:45:49.679 INFO:teuthology.lock.ops:Node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging is complete 2021-07-02T13:45:49.679 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging 2021-07-02T13:45:49.679 INFO:teuthology.lock.ops:Updating [ubuntu@smithi169.front.sepia.ceph.com]: reset os type and version on server 2021-07-02T13:45:49.680 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2021-07-02T13:45:49.699 INFO:teuthology.lock.ops:Node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging is complete 2021-07-02T13:45:49.699 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging 2021-07-02T13:45:49.700 INFO:teuthology.lock.ops:Updating [ubuntu@smithi196.front.sepia.ceph.com]: reset os type and version on server 2021-07-02T13:45:49.700 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2021-07-02T13:45:49.720 INFO:teuthology.lock.ops:Node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging is complete 2021-07-02T13:45:49.884 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of rhel 7.9 2021-07-02T13:45:49.928 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of rhel 7.9 2021-07-02T13:45:49.931 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of rhel 7.9 2021-07-02T13:45:50.294 INFO:teuthology.orchestra.console:Power off smithi112 2021-07-02T13:45:50.358 INFO:teuthology.orchestra.console:Power off smithi169 2021-07-02T13:45:50.408 INFO:teuthology.orchestra.console:Power off smithi196 2021-07-02T13:45:58.668 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2021-07-02T13:45:58.770 INFO:teuthology.orchestra.console:Power on smithi112 2021-07-02T13:45:58.798 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2021-07-02T13:45:58.899 INFO:teuthology.orchestra.console:Power on smithi196 2021-07-02T13:46:02.853 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2021-07-02T13:46:02.955 INFO:teuthology.orchestra.console:Power on smithi169 2021-07-02T13:46:07.134 INFO:teuthology.orchestra.console:Power on for smithi112 completed 2021-07-02T13:46:07.209 INFO:teuthology.orchestra.console:Power on for smithi169 completed 2021-07-02T13:46:07.235 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2021-07-02T13:46:07.295 INFO:teuthology.orchestra.console:Power on for smithi196 completed 2021-07-02T13:46:07.310 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish 2021-07-02T13:46:07.396 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2021-07-02T13:50:54.540 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-02T13:50:54.560 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-02T13:50:54.930 INFO:teuthology.provision.fog.smithi196:Node is ready 2021-07-02T13:50:54.980 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com 2021-07-02T13:50:55.017 INFO:teuthology.provision.fog.smithi112:Node is ready 2021-07-02T13:50:55.021 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196 2021-07-02T13:50:55.072 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2021-07-02T13:50:55.113 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2021-07-02T13:50:55.370 INFO:teuthology.provision.fog.smithi196:Deploy complete! 2021-07-02T13:50:55.464 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2021-07-02T13:51:09.542 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-02T13:51:09.925 INFO:teuthology.provision.fog.smithi169:Node is ready 2021-07-02T13:51:09.975 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com 2021-07-02T13:51:10.016 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2021-07-02T13:51:10.363 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2021-07-02T13:51:10.521 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2021-07-02T13:51:10.522 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-02T13:51:10.559 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2021-07-02T13:51:10.560 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-02T13:51:10.595 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com 2021-07-02T13:51:10.595 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-02T13:51:10.615 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2021-07-02T13:51:10.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-02T13:51:11.039 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2021-07-02T13:51:11.125 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Red Hat Enterprise Linux Server" 2021-07-02T13:51:11.126 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="7.9 (Maipo)" 2021-07-02T13:51:11.126 INFO:teuthology.orchestra.run.smithi112.stdout:ID="rhel" 2021-07-02T13:51:11.126 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="fedora" 2021-07-02T13:51:11.126 INFO:teuthology.orchestra.run.smithi112.stdout:VARIANT="Server" 2021-07-02T13:51:11.127 INFO:teuthology.orchestra.run.smithi112.stdout:VARIANT_ID="server" 2021-07-02T13:51:11.127 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="7.9" 2021-07-02T13:51:11.127 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Employee SKU" 2021-07-02T13:51:11.127 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2021-07-02T13:51:11.128 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server" 2021-07-02T13:51:11.128 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.redhat.com/" 2021-07-02T13:51:11.128 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-02T13:51:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: 2021-07-02T13:51:11.128 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" 2021-07-02T13:51:11.129 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=7.9 2021-07-02T13:51:11.129 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-07-02T13:51:11.129 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7.9" 2021-07-02T13:51:11.130 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2021-07-02T13:51:11.146 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2021-07-02T13:51:11.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-02T13:51:11.504 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2021-07-02T13:51:11.583 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Red Hat Enterprise Linux Server" 2021-07-02T13:51:11.583 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="7.9 (Maipo)" 2021-07-02T13:51:11.584 INFO:teuthology.orchestra.run.smithi169.stdout:ID="rhel" 2021-07-02T13:51:11.584 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="fedora" 2021-07-02T13:51:11.584 INFO:teuthology.orchestra.run.smithi169.stdout:VARIANT="Server" 2021-07-02T13:51:11.584 INFO:teuthology.orchestra.run.smithi169.stdout:VARIANT_ID="server" 2021-07-02T13:51:11.585 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="7.9" 2021-07-02T13:51:11.585 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Employee SKU" 2021-07-02T13:51:11.585 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2021-07-02T13:51:11.585 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server" 2021-07-02T13:51:11.586 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.redhat.com/" 2021-07-02T13:51:11.586 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-02T13:51:11.586 INFO:teuthology.orchestra.run.smithi169.stdout: 2021-07-02T13:51:11.586 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" 2021-07-02T13:51:11.586 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=7.9 2021-07-02T13:51:11.587 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-07-02T13:51:11.587 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7.9" 2021-07-02T13:51:11.587 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2021-07-02T13:51:11.602 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server 2021-07-02T13:51:11.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-02T13:51:11.740 ERROR:paramiko.transport:Unknown exception: The socket has already been closed by another greenlet 2021-07-02T13:51:11.742 ERROR:paramiko.transport:Traceback (most recent call last): 2021-07-02T13:51:11.743 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2055, in run 2021-07-02T13:51:11.743 ERROR:paramiko.transport: ptype, m = self.packetizer.read_message() 2021-07-02T13:51:11.743 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 459, in read_message 2021-07-02T13:51:11.744 ERROR:paramiko.transport: header = self.read_all(self.__block_size_in, check_rekey=True) 2021-07-02T13:51:11.744 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 301, in read_all 2021-07-02T13:51:11.744 ERROR:paramiko.transport: x = self.__socket.recv(n) 2021-07-02T13:51:11.745 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/virtualenv/lib/python3.6/site-packages/gevent/_socketcommon.py", line 663, in recv 2021-07-02T13:51:11.745 ERROR:paramiko.transport: self._wait(self._read_event) 2021-07-02T13:51:11.745 ERROR:paramiko.transport: File "src/gevent/_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket 2021-07-02T13:51:11.746 ERROR:paramiko.transport: File "src/gevent/_hub_primitives.py", line 321, in gevent._gevent_c_hub_primitives.wait_on_socket 2021-07-02T13:51:11.746 ERROR:paramiko.transport:gevent.exceptions.ConcurrentObjectUseError: The socket has already been closed by another greenlet 2021-07-02T13:51:11.746 ERROR:paramiko.transport: 2021-07-02T13:51:11.979 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2021-07-02T13:51:12.059 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Red Hat Enterprise Linux Server" 2021-07-02T13:51:12.059 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="7.9 (Maipo)" 2021-07-02T13:51:12.060 INFO:teuthology.orchestra.run.smithi196.stdout:ID="rhel" 2021-07-02T13:51:12.060 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="fedora" 2021-07-02T13:51:12.061 INFO:teuthology.orchestra.run.smithi196.stdout:VARIANT="Server" 2021-07-02T13:51:12.061 INFO:teuthology.orchestra.run.smithi196.stdout:VARIANT_ID="server" 2021-07-02T13:51:12.061 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="7.9" 2021-07-02T13:51:12.061 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Employee SKU" 2021-07-02T13:51:12.062 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2021-07-02T13:51:12.062 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server" 2021-07-02T13:51:12.062 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.redhat.com/" 2021-07-02T13:51:12.062 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-02T13:51:12.062 INFO:teuthology.orchestra.run.smithi196.stdout: 2021-07-02T13:51:12.063 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" 2021-07-02T13:51:12.063 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=7.9 2021-07-02T13:51:12.063 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-07-02T13:51:12.063 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7.9" 2021-07-02T13:51:12.064 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2021-07-02T13:51:12.167 INFO:teuthology.dispatcher.supervisor:Running job 6246623 2021-07-02T13:51:12.167 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-07-01_01:25:04-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6246623 --name teuthology-2021-07-01_01:25:04-upgrade:nautilus-p2p-nautilus-distro-basic-smithi --description upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} -- /home/teuthworker/archive/teuthology-2021-07-01_01:25:04-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6246623/orig.config.yaml 2021-07-02T13:51:12.171 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-07-01_01:25:04-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6246623 2021-07-02T13:51:12.172 INFO:teuthology.dispatcher.supervisor:Job PID: 14633 2021-07-02T13:51:12.174 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-07-02T14:49:17.943 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-07-02T14:49:18.169 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-07-02T14:49:18.170 INFO:teuthology.nuke:Checking targets against current locks 2021-07-02T14:49:18.240 INFO:teuthology.nuke:Not nuking smithi169.front.sepia.ceph.com because description doesn't match 2021-07-02T14:49:18.241 INFO:teuthology.nuke:Not nuking smithi112.front.sepia.ceph.com because description doesn't match 2021-07-02T14:49:18.241 INFO:teuthology.nuke:Not nuking smithi196.front.sepia.ceph.com because description doesn't match