2022-07-27T10:36:56.755 INFO:root:teuthology version: 1.1.1.dev399+g5f965ce1 2022-07-27T10:36:56.929 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging 2022-07-27T10:36:56.931 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server 2022-07-27T10:36:56.932 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-07-27T10:36:56.956 INFO:teuthology.lock.ops:Node 'smithi002.front.sepia.ceph.com' reimaging is complete 2022-07-27T10:36:56.957 INFO:teuthology.lock.ops:Start node 'smithi188.front.sepia.ceph.com' reimaging 2022-07-27T10:36:56.958 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: reset os type and version on server 2022-07-27T10:36:56.958 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2022-07-27T10:36:57.007 INFO:teuthology.lock.ops:Node 'smithi188.front.sepia.ceph.com' reimaging is complete 2022-07-27T10:36:57.209 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of rhel 8.4 2022-07-27T10:36:57.240 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of rhel 8.4 2022-07-27T10:36:57.520 INFO:teuthology.orchestra.console:Power off smithi002 2022-07-27T10:36:57.571 INFO:teuthology.orchestra.console:Power off smithi188 2022-07-27T10:36:57.689 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2022-07-27T10:36:57.724 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2022-07-27T10:36:57.791 INFO:teuthology.orchestra.console:Power on smithi002 2022-07-27T10:36:57.833 INFO:teuthology.orchestra.console:Power on smithi188 2022-07-27T10:37:06.185 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2022-07-27T10:37:06.232 INFO:teuthology.orchestra.console:Power on for smithi188 completed 2022-07-27T10:37:06.288 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2022-07-27T10:37:06.333 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish 2022-07-27T10:41:12.526 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-27T10:41:13.835 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-27T10:41:14.075 INFO:teuthology.provision.fog.smithi002:Node is ready 2022-07-27T10:41:14.091 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2022-07-27T10:41:14.148 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2022-07-27T10:41:14.481 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2022-07-27T10:41:32.989 INFO:teuthology.provision.fog.smithi188:Node is ready 2022-07-27T10:41:33.010 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188.front.sepia.ceph.com 2022-07-27T10:41:33.068 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188 2022-07-27T10:41:33.428 INFO:teuthology.provision.fog.smithi188:Deploy complete! 2022-07-27T10:41:33.661 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2022-07-27T10:41:33.662 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-27T10:41:33.683 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com 2022-07-27T10:41:33.684 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-27T10:41:33.712 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2022-07-27T10:41:33.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-07-27T10:41:33.958 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2022-07-27T10:41:34.014 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Red Hat Enterprise Linux" 2022-07-27T10:41:34.014 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8.4 (Ootpa)" 2022-07-27T10:41:34.015 INFO:teuthology.orchestra.run.smithi002.stdout:ID="rhel" 2022-07-27T10:41:34.016 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="fedora" 2022-07-27T10:41:34.016 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8.4" 2022-07-27T10:41:34.016 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2022-07-27T10:41:34.017 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-07-27T10:41:34.017 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2022-07-27T10:41:34.017 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-07-27T10:41:34.017 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.redhat.com/" 2022-07-27T10:41:34.018 INFO:teuthology.orchestra.run.smithi002.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-07-27T10:41:34.018 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-27T10:41:34.018 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-07-27T10:41:34.018 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-27T10:41:34.019 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-07-27T10:41:34.019 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-07-27T10:41:34.019 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-07-27T10:41:34.020 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-07-27T10:41:34.043 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server 2022-07-27T10:41:34.044 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2022-07-27T10:41:34.279 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2022-07-27T10:41:34.334 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="Red Hat Enterprise Linux" 2022-07-27T10:41:34.334 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8.4 (Ootpa)" 2022-07-27T10:41:34.335 INFO:teuthology.orchestra.run.smithi188.stdout:ID="rhel" 2022-07-27T10:41:34.335 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="fedora" 2022-07-27T10:41:34.335 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8.4" 2022-07-27T10:41:34.335 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8" 2022-07-27T10:41:34.336 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-07-27T10:41:34.336 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2022-07-27T10:41:34.336 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-07-27T10:41:34.336 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://www.redhat.com/" 2022-07-27T10:41:34.336 INFO:teuthology.orchestra.run.smithi188.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-07-27T10:41:34.337 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-27T10:41:34.337 INFO:teuthology.orchestra.run.smithi188.stdout: 2022-07-27T10:41:34.337 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-27T10:41:34.337 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-07-27T10:41:34.338 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-07-27T10:41:34.338 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-07-27T10:41:34.339 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2022-07-27T10:41:34.441 INFO:teuthology.dispatcher.supervisor:Running job 6949809 2022-07-27T10:41:34.441 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_5f965ce157b8dcdaf2da92a5b5d1a63e787efdd9/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2022-07-27_09:04:01-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6949809 --name gabrioux-2022-07-27_09:04:01-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi --description orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} -- /home/teuthworker/archive/gabrioux-2022-07-27_09:04:01-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6949809/orig.config.yaml 2022-07-27T10:41:34.446 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2022-07-27_09:04:01-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6949809 2022-07-27T10:41:34.447 INFO:teuthology.dispatcher.supervisor:Job PID: 24651 2022-07-27T10:41:34.447 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-27T22:42:32.999 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-27T22:42:33.520 INFO:teuthology.kill:Killing Pids: {24651} 2022-07-27T22:42:44.094 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2022-07-27T22:42:44.096 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2022-07-27T22:42:44.096 INFO:teuthology.misc:Compressing logs... 2022-07-27T22:42:44.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-07-27T22:42:44.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2022-07-27T22:42:44.768 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-07-27T22:42:44.952 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-07-27T22:42:45.046 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-07-27T22:42:45.046 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2022-07-27T22:42:45.092 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-07-27T22:42:45.092 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2022-07-27T22:42:45.096 INFO:teuthology.misc:Compressing logs... 2022-07-27T22:42:45.520 INFO:teuthology.misc:Compressing logs... 2022-07-27T22:42:47.431 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c3a0db20-0d99-11ed-8430-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-07-27T22:42:49.742 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/c3a0db20-0d99-11ed-8430-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-07-27T22:42:49.767 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c3a0db20-0d99-11ed-8430-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-07-27T22:42:53.677 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c3a0db20-0d99-11ed-8430-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-07-27T22:42:56.513 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c3a0db20-0d99-11ed-8430-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-07-27T22:43:01.902 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c3a0db20-0d99-11ed-8430-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-07-27T22:43:09.351 INFO:teuthology.kill:No teuthology processes running 2022-07-27T22:43:14.226 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi188'] 2022-07-27T22:43:20.264 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_5f965ce157b8dcdaf2da92a5b5d1a63e787efdd9/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-27T22:43:20.271 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-27T22:43:20.275 INFO:teuthology.kill:2022-07-27 22:43:20,263.263 INFO:teuthology.nuke:targets: 2022-07-27T22:43:20.276 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9YpdKf3zajdtT5afW0oRKbvF73DlwMlTCYdGcJwzxKQLo017F1CBG/FKMgSej0+DCgQ81ZuOE64BYbj6JKyb4DoypZOZdNUNmCbq9Up9L+EKxC9Sk94iIlBKYM0xPnccNlz/GzIYOBzRfCnhFwvK2mjKEDai4OT8ymI0MbFZVbC5KNerb2Q/jjIm7EDyy0aJgbYJwoz83iQdz38lUcfchlmOG72Vv1coHnD1XmR7JqU/U01Lehp59Iar6Naf78cUtxRhGQAFW2X1GXXU5yXpPKD6P9oNc8Uz8lACkLR2B776IPinA+T3riqHZ8ccAi6gsV3wjqdvc5bGqIMeciG1wR6SRXw8UU3XswziYhauta8NWopnarPPNejQbRd8WUEZ8ltpEgEjzhycQQ4mRziTDSNJGkA3+vdk5vzUZARypN2XPs6VWvN/U2yfq/2QVL1QjzYj25AZEHLFjcEp/QU8ez31QItiGSKaGQqtQaLGPd66BT3TpyrMbaBaw+g7L8+8= 2022-07-27T22:43:20.276 INFO:teuthology.kill: smithi188.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCs4g1PLB9nehsf6SEuUM71NTIvPu7ZGrVeox2jW3Qz/jJaj1McQTwlhlHQLkh667vnGsF6awl0PBlXdWH5dqAtf5CzyGjhqpZLOARpl7uZTdjTRU3wxbR0k658lK9pdTI+kvJmRrgdFrdTZLTAgS4Q02m8zhkbSd1SraFBl5utsE4Z/PxizinD+qSlbIMqp49q8mxvj4gySjmuDgDophmz1APUqBZw5GNlJxdt/fxCEMNEUmfukr/QQYNY3jbzMrZFyLFvapO1rLXXknO1QCj5bie5WVcgPYIJrVZrpO89m8kqUc0TfdrjBwWfqzgjuIOVgOEfSKWU6VujKStvWiJc2wUFiDC4crrEuixUo14Dkg4vkG0JnFilbFrAqClVJoSwesZUPgSO63LL2YCp5BT48wGJ2Zl0+AaDSXbN01zF2EAAzWe8IEK+9vx9GEaXenn84K0GB9T1GTRKw9hsnw2ZKxJkrrsDw/Ous8WnoK7TRYksnQTUUtYsPlkLWOu+of8= 2022-07-27T22:43:20.277 INFO:teuthology.kill:2022-07-27 22:43:20,263.263 INFO:teuthology.nuke:Checking targets against current locks 2022-07-27T22:43:20.360 INFO:teuthology.kill:2022-07-27 22:43:20,359.359 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-27T22:43:20.386 INFO:teuthology.kill:2022-07-27 22:43:20,386.386 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-27T22:43:20.705 INFO:teuthology.kill:2022-07-27 22:43:20,702.702 INFO:teuthology.orchestra.console:Power off smithi002 2022-07-27T22:43:20.891 INFO:teuthology.kill:2022-07-27 22:43:20,890.890 INFO:teuthology.orchestra.console:Power off smithi188 2022-07-27T22:43:29.320 INFO:teuthology.kill:2022-07-27 22:43:29,317.317 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2022-07-27T22:43:29.520 INFO:teuthology.kill:2022-07-27 22:43:29,518.518 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com 2022-07-27T22:43:33.622 INFO:teuthology.kill:2022-07-27 22:43:33,622.622 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2022-07-27T22:43:33.859 INFO:teuthology.kill:2022-07-27 22:43:33,859.859 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com 2022-07-27T22:45:35.764 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-27T22:45:36.003 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi002 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-27_18:30:47-rados-wip-yuri7-testing-2022-07-27-0808-quincy-distro-default-smithi/6950452 2022-07-27T22:45:36.004 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi188 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-27_18:30:47-rados-wip-yuri7-testing-2022-07-27-0808-quincy-distro-default-smithi/6950451