2022-09-10T10:09:51.782 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18
2022-09-10T10:09:51.958 INFO:teuthology.lock.ops:Start node 'smithi078.front.sepia.ceph.com' reimaging
2022-09-10T10:09:51.961 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: reset os type and version on server
2022-09-10T10:09:51.961 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server
2022-09-10T10:09:51.982 INFO:teuthology.lock.ops:Node 'smithi078.front.sepia.ceph.com' reimaging is complete
2022-09-10T10:09:51.982 INFO:teuthology.lock.ops:Start node 'smithi138.front.sepia.ceph.com' reimaging
2022-09-10T10:09:51.983 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: reset os type and version on server
2022-09-10T10:09:51.983 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server
2022-09-10T10:09:52.011 INFO:teuthology.lock.ops:Node 'smithi138.front.sepia.ceph.com' reimaging is complete
2022-09-10T10:09:52.213 INFO:teuthology.provision.fog.smithi078:Scheduling deploy of centos 8.stream
2022-09-10T10:09:52.250 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of centos 8.stream
2022-09-10T10:09:52.568 INFO:teuthology.orchestra.console:Power off smithi078
2022-09-10T10:09:52.617 INFO:teuthology.orchestra.console:Power off smithi138
2022-09-10T10:09:52.728 INFO:teuthology.orchestra.console:Power off for smithi078 completed
2022-09-10T10:09:52.760 INFO:teuthology.orchestra.console:Power off for smithi138 completed
2022-09-10T10:09:52.831 INFO:teuthology.orchestra.console:Power on smithi078
2022-09-10T10:09:52.865 INFO:teuthology.orchestra.console:Power on smithi138
2022-09-10T10:09:57.096 INFO:teuthology.orchestra.console:Power on for smithi078 completed
2022-09-10T10:09:57.121 INFO:teuthology.orchestra.console:Power on for smithi138 completed
2022-09-10T10:09:57.197 INFO:teuthology.provision.fog.smithi078:Waiting for deploy to finish
2022-09-10T10:09:57.223 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish
2022-09-10T10:12:28.362 ERROR:teuthology.orchestra.connection:Error authenticating with smithi138.front.sepia.ceph.com: Authentication failed.
2022-09-10T10:14:03.531 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-10T10:14:09.096 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-10T10:14:28.764 INFO:teuthology.provision.fog.smithi078:Node is ready
2022-09-10T10:14:28.809 INFO:teuthology.orchestra.run.smithi078.stdout:smithi130
2022-09-10T10:14:28.868 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.130 smithi130.front.sepia.ceph.com smithi130
2022-09-10T10:14:29.262 INFO:teuthology.provision.fog.smithi078:Deploy complete!
2022-09-10T10:14:29.366 INFO:teuthology.provision.fog.smithi138:Node is ready
2022-09-10T10:14:29.386 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com
2022-09-10T10:14:29.444 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138
2022-09-10T10:14:29.805 INFO:teuthology.provision.fog.smithi138:Deploy complete!
2022-09-10T10:14:30.057 INFO:teuthology.lock.ops:Checking smithi078.front.sepia.ceph.com
2022-09-10T10:14:30.058 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-10T10:14:30.079 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com
2022-09-10T10:14:30.080 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-10T10:14:30.114 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: set os type and version on server
2022-09-10T10:14:30.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com'
2022-09-10T10:14:30.322 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64
2022-09-10T10:14:30.380 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream"
2022-09-10T10:14:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8"
2022-09-10T10:14:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos"
2022-09-10T10:14:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora"
2022-09-10T10:14:30.382 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8"
2022-09-10T10:14:30.382 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8"
2022-09-10T10:14:30.382 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 8"
2022-09-10T10:14:30.382 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31"
2022-09-10T10:14:30.383 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-09-10T10:14:30.383 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/"
2022-09-10T10:14:30.383 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-09-10T10:14:30.383 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-09-10T10:14:30.383 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-09-10T10:14:30.385 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server
2022-09-10T10:14:30.404 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server
2022-09-10T10:14:30.404 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com'
2022-09-10T10:14:30.616 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64
2022-09-10T10:14:30.674 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream"
2022-09-10T10:14:30.675 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8"
2022-09-10T10:14:30.675 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos"
2022-09-10T10:14:30.675 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora"
2022-09-10T10:14:30.675 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8"
2022-09-10T10:14:30.676 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8"
2022-09-10T10:14:30.676 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 8"
2022-09-10T10:14:30.676 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31"
2022-09-10T10:14:30.676 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-09-10T10:14:30.677 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/"
2022-09-10T10:14:30.677 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-09-10T10:14:30.677 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-09-10T10:14:30.677 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-09-10T10:14:30.678 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server
2022-09-10T10:14:30.776 INFO:teuthology.dispatcher.supervisor:Running job 7017695
2022-09-10T10:14:30.777 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-09-07_00:21:03-orch-wip-yuri4-testing-2022-09-06-1408-distro-default-smithi/7017695 --name yuriw-2022-09-07_00:21:03-orch-wip-yuri4-testing-2022-09-06-1408-distro-default-smithi --description orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} -- /home/teuthworker/archive/yuriw-2022-09-07_00:21:03-orch-wip-yuri4-testing-2022-09-06-1408-distro-default-smithi/7017695/orig.config.yaml
2022-09-10T10:14:30.781 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-09-07_00:21:03-orch-wip-yuri4-testing-2022-09-06-1408-distro-default-smithi/7017695
2022-09-10T10:14:30.782 INFO:teuthology.dispatcher.supervisor:Job PID: 15217
2022-09-10T10:14:30.782 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-09-10T22:14:54.603 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-09-10T22:14:54.756 INFO:teuthology.kill:Killing Pids: {15217}
2022-09-10T22:14:56.273 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2022-09-10T22:14:56.274 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2022-09-10T22:14:56.274 INFO:teuthology.misc:Compressing logs...
2022-09-10T22:14:56.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com'
2022-09-10T22:14:56.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com'
2022-09-10T22:14:56.554 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-09-10T22:14:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-09-10T22:14:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-09-10T22:14:56.765 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now
2022-09-10T22:14:56.809 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-09-10T22:14:56.810 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now
2022-09-10T22:14:56.813 INFO:teuthology.misc:Compressing logs...
2022-09-10T22:14:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-09-10T22:14:57.371 INFO:teuthology.misc:Compressing logs...
2022-09-10T22:15:25.241 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-09-10T22:15:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2022-09-10T22:15:46.042 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-09-10T22:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-09-10T22:16:02.151 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2022-09-10T22:16:09.117 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-09-10T22:16:24.617 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-09-10T22:16:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-09-10T22:16:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-09-10T22:16:49.027 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2022-09-10T22:17:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/e3e95ddc-30f1-11ed-8431-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-09-10T22:17:54.252 INFO:teuthology.kill:No teuthology processes running
2022-09-10T22:17:55.749 INFO:teuthology.kill:Nuking machines: ['smithi078', 'smithi138']
2022-09-10T22:17:56.949 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-09-10T22:17:56.950 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-09-10T22:17:56.951 INFO:teuthology.kill:2022-09-10 22:17:56,949.949 INFO:teuthology.nuke:targets:
2022-09-10T22:17:56.951 INFO:teuthology.kill:  smithi078.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL9c5GbV9yxSuEdYE2n3XCFkswnW7KSyAgjfXjLcJpRy4nVg4Z09Vy3fgClAcdjF92/PLFQjHhGcV/G1gbk6otxNYf2hIHhHK/2+V0OzlMJUJ2gayC9mcMVMpFEGeaWq/gOp+fUmanHnFuK/JZ5nJtOVMedRqm4eoVvUATxTf7T7UeK844Y003UuUGgupgah9Y4g5qlQxN6nSDiGMlqL82DyZnnK7FxNPpYusDNdLIvrWgNdpG1c6b4Tu/tPyxqafDU3Zq9z2+Gcdd6Bd9aF0iRnYCnANEQ2ABkO8LPV8HiM0C65n0q41nYr5BYlcJCJLWJDFZZh8rkOBQ6U3k8jz/J3MCL77dqA5rxYzb52AV+y8sjzb7i0ciLmU62ygjNWI3ZSiGdyC3/J9RYKZZ1LQxH7Ps1Pjwv9bq/OvxQ1xMUxru8UtDKdPxMbGUKNDVfRDDkgxgJbUJLjf6ilZ2SHb9TOZrP9tCREls/sV4OKUb15n0jeWirpOpdVfby/6GZeM=
2022-09-10T22:17:56.951 INFO:teuthology.kill:  smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDDaRt24q1kQ9MRZ9rM0cDIDxri3gOnh8ezwsCrCtwPVfYgQ29mkTVzn5EPE9z9dZf314vyPsDuv8nMj1KLyiv4=
2022-09-10T22:17:56.951 INFO:teuthology.kill:2022-09-10 22:17:56,949.949 INFO:teuthology.nuke:Checking targets against current locks
2022-09-10T22:17:57.003 INFO:teuthology.kill:2022-09-10 22:17:57,003.003 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-10T22:17:57.032 INFO:teuthology.kill:2022-09-10 22:17:57,032.032 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-10T22:17:57.091 INFO:teuthology.kill:2022-09-10 22:17:57,090.090 INFO:teuthology.orchestra.console:Power off smithi078
2022-09-10T22:17:57.109 INFO:teuthology.kill:2022-09-10 22:17:57,108.108 INFO:teuthology.orchestra.console:Power off smithi138
2022-09-10T22:18:05.465 INFO:teuthology.kill:2022-09-10 22:18:05,465.465 INFO:teuthology.orchestra.console:Power off for smithi078 completed
2022-09-10T22:18:05.619 INFO:teuthology.kill:2022-09-10 22:18:05,618.618 INFO:teuthology.lock.ops:unlocked: smithi078.front.sepia.ceph.com
2022-09-10T22:18:09.579 INFO:teuthology.kill:2022-09-10 22:18:09,578.578 INFO:teuthology.orchestra.console:Power off for smithi138 completed
2022-09-10T22:18:09.736 INFO:teuthology.kill:2022-09-10 22:18:09,736.736 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com
2022-09-10T22:20:10.009 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-09-10T22:20:10.108 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi078 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-10_17:30:07-rbd-quincy-release-distro-default-smithi/7025010
2022-09-10T22:20:10.108 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi138 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-10_17:30:07-rbd-quincy-release-distro-default-smithi/7025010