2021-12-15T16:06:24.644 INFO:root:teuthology version: 1.1.0-95a7d479
2021-12-15T16:06:24.763 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi101.front.sepia.ceph.com' reimaging
2021-12-15T16:06:24.765 INFO:teuthology.lock.ops:Updating [ubuntu@smithi101.front.sepia.ceph.com]: reset os type and version on server
2021-12-15T16:06:24.765 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2021-12-15T16:06:24.784 INFO:teuthology.lock.ops:Node 'ubuntu@smithi101.front.sepia.ceph.com' reimaging is complete
2021-12-15T16:06:24.784 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging
2021-12-15T16:06:24.785 INFO:teuthology.lock.ops:Updating [ubuntu@smithi162.front.sepia.ceph.com]: reset os type and version on server
2021-12-15T16:06:24.785 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server
2021-12-15T16:06:24.804 INFO:teuthology.lock.ops:Node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging is complete
2021-12-15T16:06:24.991 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of ubuntu 20.04
2021-12-15T16:06:25.002 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of ubuntu 20.04
2021-12-15T16:06:25.291 INFO:teuthology.orchestra.console:Power off smithi101
2021-12-15T16:06:25.310 INFO:teuthology.orchestra.console:Power off smithi162
2021-12-15T16:06:33.665 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2021-12-15T16:06:33.766 INFO:teuthology.orchestra.console:Power on smithi101
2021-12-15T16:06:37.783 INFO:teuthology.orchestra.console:Power off for smithi162 completed
2021-12-15T16:06:37.884 INFO:teuthology.orchestra.console:Power on smithi162
2021-12-15T16:06:42.136 INFO:teuthology.orchestra.console:Power on for smithi101 completed
2021-12-15T16:06:42.141 INFO:teuthology.orchestra.console:Power on for smithi162 completed
2021-12-15T16:06:42.238 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish
2021-12-15T16:06:42.242 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish
2021-12-15T16:09:58.755 ERROR:teuthology.orchestra.connection:Error authenticating with smithi162.front.sepia.ceph.com: Authentication failed.
2021-12-15T16:12:15.157 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-15T16:12:15.648 INFO:teuthology.provision.fog.smithi162:Node is ready
2021-12-15T16:12:15.703 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com
2021-12-15T16:12:15.710 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162
2021-12-15T16:12:15.938 INFO:teuthology.provision.fog.smithi162:Deploy complete!
2021-12-15T16:12:30.540 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-15T16:12:31.059 INFO:teuthology.provision.fog.smithi101:Node is ready
2021-12-15T16:12:31.066 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com
2021-12-15T16:12:31.073 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101
2021-12-15T16:12:31.268 INFO:teuthology.provision.fog.smithi101:Deploy complete!
2021-12-15T16:12:31.484 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com
2021-12-15T16:12:31.485 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-15T16:12:31.504 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com
2021-12-15T16:12:31.504 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-15T16:12:31.524 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server
2021-12-15T16:12:31.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2021-12-15T16:12:31.710 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64
2021-12-15T16:12:31.760 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Ubuntu"
2021-12-15T16:12:31.760 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-12-15T16:12:31.760 INFO:teuthology.orchestra.run.smithi101.stdout:ID=ubuntu
2021-12-15T16:12:31.761 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE=debian
2021-12-15T16:12:31.761 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-12-15T16:12:31.761 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="20.04"
2021-12-15T16:12:31.761 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.ubuntu.com/"
2021-12-15T16:12:31.761 INFO:teuthology.orchestra.run.smithi101.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-12-15T16:12:31.762 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-12-15T16:12:31.762 INFO:teuthology.orchestra.run.smithi101.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-12-15T16:12:31.762 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_CODENAME=focal
2021-12-15T16:12:31.762 INFO:teuthology.orchestra.run.smithi101.stdout:UBUNTU_CODENAME=focal
2021-12-15T16:12:31.764 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2021-12-15T16:12:31.782 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server
2021-12-15T16:12:31.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com'
2021-12-15T16:12:32.050 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64
2021-12-15T16:12:32.056 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Ubuntu"
2021-12-15T16:12:32.057 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-12-15T16:12:32.057 INFO:teuthology.orchestra.run.smithi162.stdout:ID=ubuntu
2021-12-15T16:12:32.057 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE=debian
2021-12-15T16:12:32.057 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-12-15T16:12:32.058 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="20.04"
2021-12-15T16:12:32.058 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.ubuntu.com/"
2021-12-15T16:12:32.058 INFO:teuthology.orchestra.run.smithi162.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-12-15T16:12:32.058 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-12-15T16:12:32.058 INFO:teuthology.orchestra.run.smithi162.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-12-15T16:12:32.059 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_CODENAME=focal
2021-12-15T16:12:32.059 INFO:teuthology.orchestra.run.smithi162.stdout:UBUNTU_CODENAME=focal
2021-12-15T16:12:32.059 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server
2021-12-15T16:12:32.175 INFO:teuthology.dispatcher.supervisor:Running job 6564223
2021-12-15T16:12:32.176 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-12-15_15:59:44-rados-wip-sage2-testing-2021-12-14-1714-distro-basic-smithi/6564223 --name sage-2021-12-15_15:59:44-rados-wip-sage2-testing-2021-12-14-1714-distro-basic-smithi --description rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} -- /home/teuthworker/archive/sage-2021-12-15_15:59:44-rados-wip-sage2-testing-2021-12-14-1714-distro-basic-smithi/6564223/orig.config.yaml
2021-12-15T16:12:32.180 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-12-15_15:59:44-rados-wip-sage2-testing-2021-12-14-1714-distro-basic-smithi/6564223
2021-12-15T16:12:32.181 INFO:teuthology.dispatcher.supervisor:Job PID: 9569
2021-12-15T16:12:32.181 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-16T04:12:53.512 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-16T04:12:53.694 INFO:teuthology.kill:Killing Pids: {9569}
2021-12-16T04:12:54.903 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a']
2021-12-16T04:12:54.904 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2021-12-16T04:12:54.904 INFO:teuthology.misc:Compressing logs...
2021-12-16T04:12:54.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2021-12-16T04:12:55.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com'
2021-12-16T04:12:55.332 INFO:teuthology.misc:Compressing logs...
2021-12-16T04:12:55.920 INFO:teuthology.misc:Compressing logs...
2021-12-16T04:12:59.422 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-mgr.x.log: file size changed while zipping
2021-12-16T04:13:11.653 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-16T04:13:16.367 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-16T04:13:31.840 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-16T04:13:32.331 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2021-12-16T04:13:36.460 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-16T04:13:56.410 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-16T04:14:00.358 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-16T04:14:06.853 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2021-12-16T04:14:16.773 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-16T04:14:24.045 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2021-12-16T04:14:30.262 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-16T04:14:45.974 INFO:teuthology.orchestra.run.smithi101.stderr:gzip: /var/log/ceph/0ceb952c-5dc3-11ec-8c31-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2021-12-16T04:15:36.566 INFO:teuthology.kill:No teuthology processes running
2021-12-16T04:15:37.704 INFO:teuthology.kill:Nuking machines: ['smithi101', 'smithi162']
2021-12-16T04:15:38.688 INFO:teuthology.kill:2021-12-16 04:15:38,687.687 INFO:teuthology.nuke:targets:
2021-12-16T04:15:38.689 INFO:teuthology.kill:  smithi101.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwcLszYaxR883dKPmwtognzh5bLeAkSbumI9mSnd6Dwm0tW8yE9XqP9K+oETlMElzSpsE4ekimpIdC+n2nrKf3BdFMreoM7AWo14ic0UKQsohWi/s8epqHn/bVPKDWM7pRIOwbBtPl7bSTJiGEXqAIlYX25R/FgDTk/DiDJbdsaOcC4Z8OU8BuxhYAqOKYjothUH/xK7+PYFG72ZVIZqTQeOxcdZqc7ECRDEQY5PqJqcpBgv9iIT5tu2nQkCmTwfHU9oTDO7IRZqXw4UcjZAMPk35Kauih9eMkMPStJR+DfggUJssWGWcEYge9cjd+GSDKysobZFlojcybnJRuCqjyQ2JBjy5wWhN/wXOgsxBnY0dZOaZc6Nz4x7zcrwoy4JC9gFrXNBq3F+Lq6rn7dnMV0KbD528F5+3mPth6rnjcvJaEO7K/x5RA5ZHi59qi7BTBUy2dT1KtupZKzSHlXgzhAC0v1gEGYOPHyDcMy/A/TRvBI5wHbYdxfS1DYAmOJek=
2021-12-16T04:15:38.689 INFO:teuthology.kill:  smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHa4PSBq9H0O6skbdoEofcN70AxgFEBsOFfSYSSLb0OTi4jASKi/QysEizaqw2v0kBqUx4qMyrHvPJ7fEiWyAc5FTEk9K9VCvBY7yWCDX2cGbt7iCnwhSdj5MNfD+S3SdTXnu0G9gv3bUZLVa3EGCYzVoncV1UD9Lx6BsgW840yLfgFthcLTyO5p7FIFQtV0N/ky2z9eXQ/DIrw7HvEmSxc9gXA636XT1ydpc9yR5zS8qByauS9SqJdS2qqBzNP38z/8ZoCRKE5HkXevtv3VkA4r0pGwn0ceFdGFcHlVuJiigtaYNNBr3QZOuPqjNCdjiBr87DMZKGw0YtkVHGJQDXQ9Eh/HL3UPGO+zAiKqwGXRFFRLoA1oOMpACX96yX5USCxbooPEy9LxzGPqbNve05V32mz+H9xpYFX9WHEtiqP+1uXA+xnf/2xmG75n1JejkmsuZsJAGtZZWj9li0TPE824AxuBZ0XaleN4ku0SL0u/vLn/ynARzRIOXYYnFdgsM=
2021-12-16T04:15:38.716 INFO:teuthology.kill:2021-12-16 04:15:38,715.715 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-16T04:15:38.719 INFO:teuthology.kill:2021-12-16 04:15:38,719.719 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-16T04:15:38.802 INFO:teuthology.kill:2021-12-16 04:15:38,802.802 INFO:teuthology.orchestra.console:Power off smithi101
2021-12-16T04:15:38.842 INFO:teuthology.kill:2021-12-16 04:15:38,841.841 INFO:teuthology.orchestra.console:Power off smithi162
2021-12-16T04:15:47.178 INFO:teuthology.kill:2021-12-16 04:15:47,178.178 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2021-12-16T04:15:47.325 INFO:teuthology.kill:2021-12-16 04:15:47,324.324 INFO:teuthology.lock.ops:unlocked: smithi101.front.sepia.ceph.com
2021-12-16T04:15:51.318 INFO:teuthology.kill:2021-12-16 04:15:51,316.316 INFO:teuthology.orchestra.console:Power off for smithi162 completed
2021-12-16T04:15:51.460 INFO:teuthology.kill:2021-12-16 04:15:51,460.460 INFO:teuthology.lock.ops:unlocked: smithi162.front.sepia.ceph.com
2021-12-16T04:17:51.683 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-16T04:17:51.765 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi101 but it was locked by another job: /home/teuthworker/archive/akupczyk-2021-12-15_18:10:21-rados-wip-bluefs-fine-grain-locking-4-rebase-distro-default-smithi/6565209
2021-12-16T04:17:51.765 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi162 but it was locked by another job: /home/teuthworker/archive/akupczyk-2021-12-15_18:10:21-rados-wip-bluefs-fine-grain-locking-4-rebase-distro-default-smithi/6565209