2022-08-15T15:01:58.629 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc
2022-08-15T15:01:58.790 INFO:teuthology.lock.ops:Start node 'smithi063.front.sepia.ceph.com' reimaging
2022-08-15T15:01:58.792 INFO:teuthology.lock.ops:Updating [smithi063.front.sepia.ceph.com]: reset os type and version on server
2022-08-15T15:01:58.792 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server
2022-08-15T15:01:58.816 INFO:teuthology.lock.ops:Node 'smithi063.front.sepia.ceph.com' reimaging is complete
2022-08-15T15:01:58.817 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging
2022-08-15T15:01:58.817 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server
2022-08-15T15:01:58.818 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2022-08-15T15:01:58.838 INFO:teuthology.lock.ops:Node 'smithi149.front.sepia.ceph.com' reimaging is complete
2022-08-15T15:01:59.027 INFO:teuthology.provision.fog.smithi063:Scheduling deploy of rhel 8.6
2022-08-15T15:01:59.045 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of rhel 8.6
2022-08-15T15:01:59.349 INFO:teuthology.orchestra.console:Power off smithi149
2022-08-15T15:01:59.367 INFO:teuthology.orchestra.console:Power off smithi063
2022-08-15T15:01:59.497 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2022-08-15T15:01:59.598 INFO:teuthology.orchestra.console:Power on smithi149
2022-08-15T15:02:08.016 INFO:teuthology.orchestra.console:Power on for smithi149 completed
2022-08-15T15:02:08.118 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish
2022-08-15T15:02:11.851 INFO:teuthology.orchestra.console:Power off for smithi063 completed
2022-08-15T15:02:11.952 INFO:teuthology.orchestra.console:Power on smithi063
2022-08-15T15:02:20.314 INFO:teuthology.orchestra.console:Power on for smithi063 completed
2022-08-15T15:02:20.415 INFO:teuthology.provision.fog.smithi063:Waiting for deploy to finish
2022-08-15T15:04:51.526 ERROR:teuthology.orchestra.connection:Error authenticating with smithi063.front.sepia.ceph.com: Authentication failed.
2022-08-15T15:06:55.154 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-08-15T15:06:55.379 INFO:teuthology.provision.fog.smithi149:Node is ready
2022-08-15T15:06:55.436 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149
2022-08-15T15:06:55.493 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149
2022-08-15T15:06:55.810 INFO:teuthology.provision.fog.smithi149:Deploy complete!
2022-08-15T15:07:04.821 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-08-15T15:07:05.076 INFO:teuthology.provision.fog.smithi063:Node is ready
2022-08-15T15:07:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063
2022-08-15T15:07:05.200 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63 smithi063.front.sepia.ceph.com smithi063
2022-08-15T15:07:05.556 INFO:teuthology.provision.fog.smithi063:Deploy complete!
2022-08-15T15:07:05.796 INFO:teuthology.lock.ops:Checking smithi063.front.sepia.ceph.com
2022-08-15T15:07:05.798 INFO:teuthology.lock.ops:New key found. Updating...
2022-08-15T15:07:05.824 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com
2022-08-15T15:07:05.824 INFO:teuthology.lock.ops:New key found. Updating...
2022-08-15T15:07:05.844 INFO:teuthology.lock.ops:Updating [smithi063.front.sepia.ceph.com]: set os type and version on server
2022-08-15T15:07:05.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com'
2022-08-15T15:07:06.048 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64
2022-08-15T15:07:06.108 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="Red Hat Enterprise Linux"
2022-08-15T15:07:06.109 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="8.6 (Ootpa)"
2022-08-15T15:07:06.109 INFO:teuthology.orchestra.run.smithi063.stdout:ID="rhel"
2022-08-15T15:07:06.109 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="fedora"
2022-08-15T15:07:06.110 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="8.6"
2022-08-15T15:07:06.110 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el8"
2022-08-15T15:07:06.110 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-08-15T15:07:06.110 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31"
2022-08-15T15:07:06.111 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-08-15T15:07:06.111 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://www.redhat.com/"
2022-08-15T15:07:06.111 INFO:teuthology.orchestra.run.smithi063.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-08-15T15:07:06.111 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-08-15T15:07:06.112 INFO:teuthology.orchestra.run.smithi063.stdout:
2022-08-15T15:07:06.112 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-08-15T15:07:06.112 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-08-15T15:07:06.112 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-08-15T15:07:06.112 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-08-15T15:07:06.114 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server
2022-08-15T15:07:06.141 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server
2022-08-15T15:07:06.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2022-08-15T15:07:06.348 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64
2022-08-15T15:07:06.407 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Red Hat Enterprise Linux"
2022-08-15T15:07:06.407 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8.6 (Ootpa)"
2022-08-15T15:07:06.408 INFO:teuthology.orchestra.run.smithi149.stdout:ID="rhel"
2022-08-15T15:07:06.408 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="fedora"
2022-08-15T15:07:06.408 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8.6"
2022-08-15T15:07:06.408 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8"
2022-08-15T15:07:06.409 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-08-15T15:07:06.409 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2022-08-15T15:07:06.409 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-08-15T15:07:06.409 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.redhat.com/"
2022-08-15T15:07:06.409 INFO:teuthology.orchestra.run.smithi149.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-08-15T15:07:06.410 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-08-15T15:07:06.410 INFO:teuthology.orchestra.run.smithi149.stdout:
2022-08-15T15:07:06.410 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-08-15T15:07:06.410 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-08-15T15:07:06.411 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-08-15T15:07:06.411 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-08-15T15:07:06.411 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2022-08-15T15:07:06.507 INFO:teuthology.dispatcher.supervisor:Running job 6973466
2022-08-15T15:07:06.507 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_9e7483cc68a9eb6b54dacbb0bec3bf23a5d32425/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-08-15_13:41:30-orch-wip-yuri8-testing-2022-08-12-0833-distro-default-smithi/6973466 --name yuriw-2022-08-15_13:41:30-orch-wip-yuri8-testing-2022-08-12-0833-distro-default-smithi --description orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} -- /home/teuthworker/archive/yuriw-2022-08-15_13:41:30-orch-wip-yuri8-testing-2022-08-12-0833-distro-default-smithi/6973466/orig.config.yaml
2022-08-15T15:07:06.512 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-08-15_13:41:30-orch-wip-yuri8-testing-2022-08-12-0833-distro-default-smithi/6973466
2022-08-15T15:07:06.512 INFO:teuthology.dispatcher.supervisor:Job PID: 28139
2022-08-15T15:07:06.513 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-08-16T03:07:29.505 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-08-16T03:07:29.756 INFO:teuthology.kill:Killing Pids: {28139}
2022-08-16T03:07:32.118 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a']
2022-08-16T03:07:32.119 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2022-08-16T03:07:32.120 INFO:teuthology.misc:Compressing logs...
2022-08-16T03:07:32.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com'
2022-08-16T03:07:32.402 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2022-08-16T03:07:33.076 INFO:teuthology.misc:Compressing logs...
2022-08-16T03:07:33.288 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-08-16T03:07:33.497 INFO:teuthology.misc:Compressing logs...
2022-08-16T03:07:51.283 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-08-16T03:07:53.734 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-08-16T03:08:11.600 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-08-16T03:08:13.386 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-08-16T03:08:35.743 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-08-16T03:08:39.837 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-mgr.x.log: file size changed while zipping
2022-08-16T03:08:39.876 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-08-16T03:09:02.339 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-08-16T03:09:04.954 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-08-16T03:09:14.745 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2022-08-16T03:09:17.612 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2022-08-16T03:09:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/a5782662-1cad-11ed-8431-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2022-08-16T03:10:49.336 INFO:teuthology.kill:No teuthology processes running
2022-08-16T03:10:51.632 INFO:teuthology.kill:Nuking machines: ['smithi063', 'smithi149']
2022-08-16T03:10:53.372 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_9e7483cc68a9eb6b54dacbb0bec3bf23a5d32425/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-08-16T03:10:53.373 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-08-16T03:10:53.373 INFO:teuthology.kill:2022-08-16 03:10:53,372.372 INFO:teuthology.nuke:targets:
2022-08-16T03:10:53.374 INFO:teuthology.kill:  smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIBbFjJDmTdbeyZpNZbGklY5oJ8dkRh1BHP9+D0KC5bXprJHvluU33RT+JaCichYiKLhPPjL4K8NLzkwzCbYdI=
2022-08-16T03:10:53.374 INFO:teuthology.kill:  smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfDZNR6Kw18nEzzNNuQw2CrlenbxM0puwMOSKqAynrK23lL4wDFo33FVnHS4BLlrtS5C8lFPdXAJv97Vb9oXlWjpjqtcMHmKt+AFi/aFEch/AiSBgxlrRrJ1jPnSn5CPz0ARxDUGsZkV8NXyXKFaUmqrLYicHq4zsXZ8nFSbTJRxeJlPEYWtM58hyO2x2yGOliX07bVQoUv0BOcpeKlR6eu0kio4HWyEceogCuyD+V1I1+ESVAbEtYxJwx9zyZhxUroUtA1KmUME1sE9q0wpFQWIEvCzwcUA9rF1adI4VTsRShBt7NWrvmygJFQdmx9LIQjZLzgN6k/QOPr8LpVhiraCAba4kY0/916bKpZr2idjJEN3gRWH0In1jUxvR9FBMVcji8oqgmkiKt+QBMJpH6L6eDqb7kfPgGmr+/GK2oO1HbRmvQ4h928sB3q+GSOVFxWzqxsnFE+r3bjehtWq7W0yuTGtGC7liL1B513ILvO4z512W+lEofGIyWMsuRMr0=
2022-08-16T03:10:53.375 INFO:teuthology.kill:2022-08-16 03:10:53,372.372 INFO:teuthology.nuke:Checking targets against current locks
2022-08-16T03:10:53.438 INFO:teuthology.kill:2022-08-16 03:10:53,438.438 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-08-16T03:10:53.468 INFO:teuthology.kill:2022-08-16 03:10:53,468.468 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-08-16T03:10:53.571 INFO:teuthology.kill:2022-08-16 03:10:53,570.570 INFO:teuthology.orchestra.console:Power off smithi063
2022-08-16T03:10:53.654 INFO:teuthology.kill:2022-08-16 03:10:53,653.653 INFO:teuthology.orchestra.console:Power off smithi149
2022-08-16T03:11:02.028 INFO:teuthology.kill:2022-08-16 03:11:02,027.027 INFO:teuthology.orchestra.console:Power off for smithi063 completed
2022-08-16T03:11:02.225 INFO:teuthology.kill:2022-08-16 03:11:02,224.224 INFO:teuthology.lock.ops:unlocked: smithi063.front.sepia.ceph.com
2022-08-16T03:11:06.262 INFO:teuthology.kill:2022-08-16 03:11:06,261.261 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2022-08-16T03:11:06.427 INFO:teuthology.kill:2022-08-16 03:11:06,418.418 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com
2022-08-16T03:13:06.865 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-08-16T03:13:07.157 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi063 but it was locked by another job: /home/teuthworker/archive/amaredia-2022-08-16_00:56:18-rgw:verify-wip-rgw-timeout-fail-check-6-8-22-distro-default-smithi/6974350
2022-08-16T03:13:07.158 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi149 but it was locked by another job: /home/teuthworker/archive/amaredia-2022-08-16_00:56:18-rgw:verify-wip-rgw-timeout-fail-check-6-8-22-distro-default-smithi/6974348