2022-02-03T22:23:16.151 INFO:root:teuthology version: 1.1.0-3094160c
2022-02-03T22:23:16.277 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi008.front.sepia.ceph.com' reimaging
2022-02-03T22:23:16.278 INFO:teuthology.lock.ops:Updating [ubuntu@smithi008.front.sepia.ceph.com]: reset os type and version on server
2022-02-03T22:23:16.279 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server
2022-02-03T22:23:16.301 INFO:teuthology.lock.ops:Node 'ubuntu@smithi008.front.sepia.ceph.com' reimaging is complete
2022-02-03T22:23:16.302 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi175.front.sepia.ceph.com' reimaging
2022-02-03T22:23:16.302 INFO:teuthology.lock.ops:Updating [ubuntu@smithi175.front.sepia.ceph.com]: reset os type and version on server
2022-02-03T22:23:16.303 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server
2022-02-03T22:23:16.321 INFO:teuthology.lock.ops:Node 'ubuntu@smithi175.front.sepia.ceph.com' reimaging is complete
2022-02-03T22:23:16.529 INFO:teuthology.provision.fog.smithi175:Scheduling deploy of ubuntu 20.04
2022-02-03T22:23:16.532 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of ubuntu 20.04
2022-02-03T22:23:16.891 INFO:teuthology.orchestra.console:Power off smithi175
2022-02-03T22:23:16.951 INFO:teuthology.orchestra.console:Power off smithi008
2022-02-03T22:23:17.035 INFO:teuthology.orchestra.console:Power off for smithi175 completed
2022-02-03T22:23:17.093 INFO:teuthology.orchestra.console:Power off for smithi008 completed
2022-02-03T22:23:17.137 INFO:teuthology.orchestra.console:Power on smithi175
2022-02-03T22:23:17.195 INFO:teuthology.orchestra.console:Power on smithi008
2022-02-03T22:23:21.455 INFO:teuthology.orchestra.console:Power on for smithi008 completed
2022-02-03T22:23:21.556 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish
2022-02-03T22:23:25.512 INFO:teuthology.orchestra.console:Power on for smithi175 completed
2022-02-03T22:23:25.613 INFO:teuthology.provision.fog.smithi175:Waiting for deploy to finish
2022-02-03T22:26:12.076 ERROR:teuthology.orchestra.connection:Error authenticating with smithi175.front.sepia.ceph.com: Authentication failed.
2022-02-03T22:28:14.627 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-03T22:28:14.729 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-03T22:28:15.120 INFO:teuthology.provision.fog.smithi008:Node is ready
2022-02-03T22:28:15.127 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com
2022-02-03T22:28:15.178 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008
2022-02-03T22:28:15.193 INFO:teuthology.provision.fog.smithi175:Node is ready
2022-02-03T22:28:15.242 INFO:teuthology.orchestra.run.smithi175.stdout:smithi175.front.sepia.ceph.com
2022-02-03T22:28:15.249 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175 smithi175.front.sepia.ceph.com smithi175
2022-02-03T22:28:15.364 INFO:teuthology.provision.fog.smithi008:Deploy complete!
2022-02-03T22:28:15.395 INFO:teuthology.provision.fog.smithi175:Deploy complete!
2022-02-03T22:28:15.611 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com
2022-02-03T22:28:15.612 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-03T22:28:15.631 INFO:teuthology.lock.ops:Checking smithi175.front.sepia.ceph.com
2022-02-03T22:28:15.631 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-03T22:28:15.649 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server
2022-02-03T22:28:15.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2022-02-03T22:28:15.835 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64
2022-02-03T22:28:15.885 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="Ubuntu"
2022-02-03T22:28:15.886 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-03T22:28:15.886 INFO:teuthology.orchestra.run.smithi008.stdout:ID=ubuntu
2022-02-03T22:28:15.886 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE=debian
2022-02-03T22:28:15.886 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-03T22:28:15.887 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="20.04"
2022-02-03T22:28:15.887 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-03T22:28:15.887 INFO:teuthology.orchestra.run.smithi008.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-03T22:28:15.887 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-03T22:28:15.888 INFO:teuthology.orchestra.run.smithi008.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-03T22:28:15.888 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_CODENAME=focal
2022-02-03T22:28:15.888 INFO:teuthology.orchestra.run.smithi008.stdout:UBUNTU_CODENAME=focal
2022-02-03T22:28:15.889 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server
2022-02-03T22:28:15.908 INFO:teuthology.lock.ops:Updating [smithi175.front.sepia.ceph.com]: set os type and version on server
2022-02-03T22:28:15.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com'
2022-02-03T22:28:16.082 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64
2022-02-03T22:28:16.088 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Ubuntu"
2022-02-03T22:28:16.089 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-03T22:28:16.089 INFO:teuthology.orchestra.run.smithi175.stdout:ID=ubuntu
2022-02-03T22:28:16.089 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE=debian
2022-02-03T22:28:16.089 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-03T22:28:16.090 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="20.04"
2022-02-03T22:28:16.090 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-03T22:28:16.090 INFO:teuthology.orchestra.run.smithi175.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-03T22:28:16.090 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-03T22:28:16.091 INFO:teuthology.orchestra.run.smithi175.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-03T22:28:16.091 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_CODENAME=focal
2022-02-03T22:28:16.091 INFO:teuthology.orchestra.run.smithi175.stdout:UBUNTU_CODENAME=focal
2022-02-03T22:28:16.092 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server
2022-02-03T22:28:16.181 INFO:teuthology.dispatcher.supervisor:Running job 6659645
2022-02-03T22:28:16.181 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-02-03_22:11:02-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6659645 --name adking-2022-02-03_22:11:02-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi --description orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} -- /home/teuthworker/archive/adking-2022-02-03_22:11:02-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6659645/orig.config.yaml
2022-02-03T22:28:16.184 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-02-03_22:11:02-orch:cephadm-wip-adk-testing-2022-02-03-0853-distro-basic-smithi/6659645
2022-02-03T22:28:16.185 INFO:teuthology.dispatcher.supervisor:Job PID: 716
2022-02-03T22:28:16.186 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-04T04:58:27.145 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-04T04:58:27.314 INFO:teuthology.kill:Killing Pids: {716}
2022-02-04T04:58:28.562 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2022-02-04T04:58:28.563 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2022-02-04T04:58:28.563 INFO:teuthology.misc:Compressing logs...
2022-02-04T04:58:28.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2022-02-04T04:58:28.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com'
2022-02-04T04:58:28.765 INFO:teuthology.orchestra.run.smithi008.stderr:find: '/var/lib/ceph/crash': No such file or directory
2022-02-04T04:58:28.985 INFO:teuthology.orchestra.run.smithi175.stderr:find: '/var/lib/ceph/crash': No such file or directory
2022-02-04T04:58:29.017 INFO:teuthology.orchestra.run.smithi008.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-04T04:58:29.018 INFO:teuthology.orchestra.run.smithi008.stderr:tar: Error is not recoverable: exiting now
2022-02-04T04:58:29.035 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-04T04:58:29.035 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now
2022-02-04T04:58:29.037 INFO:teuthology.misc:Compressing logs...
2022-02-04T04:58:29.409 INFO:teuthology.misc:Compressing logs...
2022-02-04T04:58:32.042 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2022-02-04T04:58:44.308 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-04T04:58:49.034 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-04T04:58:53.576 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2022-02-04T04:58:57.578 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-04T04:59:01.479 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2022-02-04T04:59:07.112 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2022-02-04T04:59:11.645 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-04T04:59:22.078 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-04T04:59:23.218 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-04T04:59:36.985 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-04T04:59:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/0492a7a8-8541-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-04T05:00:20.626 INFO:teuthology.kill:No teuthology processes running
2022-02-04T05:00:21.866 INFO:teuthology.kill:Nuking machines: ['smithi008', 'smithi175']
2022-02-04T05:00:22.846 INFO:teuthology.kill:2022-02-04 05:00:22,846.846 INFO:teuthology.nuke:targets:
2022-02-04T05:00:22.847 INFO:teuthology.kill:  smithi008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAiflAHfOGi67cyRGckEOLnFZC9hsDop1s1ct25ZRMfV3FqfbZ2ZFtaKd7e+iIyQZKaaZ4MZgrOMLwnj9fuEhkH/hp927VbuwcgU90Q9kApUyq6d4AcEY7iHHFQ+TtroGWGr8vTcC6Rh9hbNr4CQq2oHnVtFnl9oFZA0G6bin3oeSuTJM0fg20UB2PXrky3QHPf8/0MmbnexHcQYf8qNvrVykVW0xJgshfYvuToJ/QS3Zq/PM6bcU/aV3ciNPGSgcEyLuYD6B42lY54rpTjr8TaEhl/l9OamxMn9MWKtR/+D/e3E3rZH/wh3UGgwfcwhshlbFfAEI3tWWa35fpXNgRxbm6tV1cJYeZl4IPbSmGTXizLR4IbCY8zgerbIQZgipwLInt3gjmEh90JihrOajMv8EmYL0RxRiM/Qq1hi+MMvyygLCmgL6v/WXFoCqZu4iVLMYjUCupc/k7a0erPCLzLHKFYCNwHhwVyuBLlWZWH5x6XgG7nB1qWJVG4Vd4R9U=
2022-02-04T05:00:22.847 INFO:teuthology.kill:  smithi175.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUVaNqUHQi9B5+NTaQG7qshAuH2YglHsWBA8M8Ci6N6Yao82IhLeKgXe/w667IQRS7DF5BOcKFpYz+JbP0Sqq2vC/lZql5FYLiilPu2fZLzDBEbhObrcAjQxur3+cjq1lMsPIW5rH83dKP/gEHhG6KEhFsFsnMcBbSb8GOqW6As+of6tZ07nXobXL+eFwZkQWmE7ddPHEBbLUTXjdIJ0WMVE0Sy0kVDxjYjFIkwlvhDcciI4qENfsKrp60ihJ2n3bA5Jhp69LBd0eh3VodLbSW8O5z5Wzyx8ixyJ+QohpSO/dj9RPUq98L4DDLIZW+nNdK0FvdmLp/QPp6N75s7X5HzbMl4UgTV7/f3/VawMMedvxGZehbqIhO+T5u1fvAoWbhx5KkGvAzR4Cufr9gqxIzMVGZWdDg3yeRL8p2gv4gyRkQfNbsqo55etiNEJi8Uu7JBD2IWET/GnWvOOFvH0u3vNR3lrDELuxI8C7/xzOzd5uBQHRpkgyo5UmLvRfvUk8=
2022-02-04T05:00:22.875 INFO:teuthology.kill:2022-02-04 05:00:22,874.874 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-04T05:00:22.878 INFO:teuthology.kill:2022-02-04 05:00:22,878.878 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-04T05:00:22.955 INFO:teuthology.kill:2022-02-04 05:00:22,955.955 INFO:teuthology.orchestra.console:Power off smithi008
2022-02-04T05:00:22.969 INFO:teuthology.kill:2022-02-04 05:00:22,968.968 INFO:teuthology.orchestra.console:Power off smithi175
2022-02-04T05:00:31.336 INFO:teuthology.kill:2022-02-04 05:00:31,335.335 INFO:teuthology.orchestra.console:Power off for smithi175 completed
2022-02-04T05:00:31.483 INFO:teuthology.kill:2022-02-04 05:00:31,482.482 INFO:teuthology.lock.ops:unlocked: smithi175.front.sepia.ceph.com
2022-02-04T05:00:35.441 INFO:teuthology.kill:2022-02-04 05:00:35,441.441 INFO:teuthology.orchestra.console:Power off for smithi008 completed
2022-02-04T05:00:35.590 INFO:teuthology.kill:2022-02-04 05:00:35,589.589 INFO:teuthology.lock.ops:unlocked: smithi008.front.sepia.ceph.com
2022-02-04T05:02:35.820 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-04T05:02:35.906 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi008 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-04_00:05:59-rados-wip-yuri-testing-2022-02-03-1333-distro-default-smithi/6660269
2022-02-04T05:02:35.907 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi175 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-04_00:05:59-rados-wip-yuri-testing-2022-02-03-1333-distro-default-smithi/6660271