2022-07-28T21:05:20.294 INFO:root:teuthology version: 1.1.1.dev399+g5f965ce1 2022-07-28T21:05:20.448 INFO:teuthology.lock.ops:Start node 'smithi037.front.sepia.ceph.com' reimaging 2022-07-28T21:05:20.450 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: reset os type and version on server 2022-07-28T21:05:20.451 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2022-07-28T21:05:20.479 INFO:teuthology.lock.ops:Node 'smithi037.front.sepia.ceph.com' reimaging is complete 2022-07-28T21:05:20.480 INFO:teuthology.lock.ops:Start node 'smithi138.front.sepia.ceph.com' reimaging 2022-07-28T21:05:20.480 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: reset os type and version on server 2022-07-28T21:05:20.480 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-07-28T21:05:20.501 INFO:teuthology.lock.ops:Node 'smithi138.front.sepia.ceph.com' reimaging is complete 2022-07-28T21:05:20.697 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of centos 8.stream 2022-07-28T21:05:20.743 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of centos 8.stream 2022-07-28T21:05:21.121 INFO:teuthology.orchestra.console:Power off smithi037 2022-07-28T21:05:21.199 INFO:teuthology.orchestra.console:Power off smithi138 2022-07-28T21:05:21.273 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2022-07-28T21:05:21.342 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-07-28T21:05:21.378 INFO:teuthology.orchestra.console:Power on smithi037 2022-07-28T21:05:21.443 INFO:teuthology.orchestra.console:Power on smithi138 2022-07-28T21:05:25.641 INFO:teuthology.orchestra.console:Power on for smithi037 completed 2022-07-28T21:05:25.742 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish 2022-07-28T21:05:29.826 INFO:teuthology.orchestra.console:Power on for smithi138 completed 2022-07-28T21:05:29.927 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2022-07-28T21:09:33.916 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-28T21:09:34.120 INFO:teuthology.provision.fog.smithi037:Node is ready 2022-07-28T21:09:34.180 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com 2022-07-28T21:09:34.240 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037 2022-07-28T21:09:34.603 INFO:teuthology.provision.fog.smithi037:Deploy complete! 2022-07-28T21:09:42.785 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-28T21:10:07.640 INFO:teuthology.provision.fog.smithi138:Node is ready 2022-07-28T21:10:07.665 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2022-07-28T21:10:07.751 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2022-07-28T21:10:08.131 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2022-07-28T21:10:08.413 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com 2022-07-28T21:10:08.415 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-28T21:10:08.437 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2022-07-28T21:10:08.438 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-28T21:10:08.483 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server 2022-07-28T21:10:08.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2022-07-28T21:10:08.851 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2022-07-28T21:10:08.912 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2022-07-28T21:10:08.912 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8" 2022-07-28T21:10:08.913 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2022-07-28T21:10:08.913 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2022-07-28T21:10:08.913 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8" 2022-07-28T21:10:08.913 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8" 2022-07-28T21:10:08.914 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-28T21:10:08.914 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2022-07-28T21:10:08.914 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-28T21:10:08.915 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2022-07-28T21:10:08.915 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-28T21:10:08.915 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-28T21:10:08.915 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-28T21:10:08.919 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2022-07-28T21:10:08.945 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2022-07-28T21:10:08.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-07-28T21:10:09.172 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2022-07-28T21:10:09.231 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2022-07-28T21:10:09.232 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8" 2022-07-28T21:10:09.232 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2022-07-28T21:10:09.232 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2022-07-28T21:10:09.233 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2022-07-28T21:10:09.233 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2022-07-28T21:10:09.234 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-28T21:10:09.235 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2022-07-28T21:10:09.235 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-28T21:10:09.235 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2022-07-28T21:10:09.236 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-28T21:10:09.236 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-28T21:10:09.236 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-28T21:10:09.238 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-07-28T21:10:09.364 INFO:teuthology.dispatcher.supervisor:Running job 6952626 2022-07-28T21:10:09.364 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_5f965ce157b8dcdaf2da92a5b5d1a63e787efdd9/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-07-28_15:19:26-rados-wip-yuri-testing-2022-07-25-0645-quincy-distro-default-smithi/6952626 --name yuriw-2022-07-28_15:19:26-rados-wip-yuri-testing-2022-07-25-0645-quincy-distro-default-smithi --description rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} -- /home/teuthworker/archive/yuriw-2022-07-28_15:19:26-rados-wip-yuri-testing-2022-07-25-0645-quincy-distro-default-smithi/6952626/orig.config.yaml 2022-07-28T21:10:09.373 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-07-28_15:19:26-rados-wip-yuri-testing-2022-07-25-0645-quincy-distro-default-smithi/6952626 2022-07-28T21:10:09.374 INFO:teuthology.dispatcher.supervisor:Job PID: 18733 2022-07-28T21:10:09.374 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-29T09:11:04.040 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-29T09:11:04.195 INFO:teuthology.kill:Killing Pids: {18733} 2022-07-29T09:11:05.691 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2022-07-29T09:11:05.692 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2022-07-29T09:11:05.693 INFO:teuthology.misc:Compressing logs... 2022-07-29T09:11:05.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2022-07-29T09:11:05.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-07-29T09:11:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-07-29T09:11:06.117 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-07-29T09:11:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-07-29T09:11:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2022-07-29T09:11:06.196 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-07-29T09:11:06.196 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2022-07-29T09:11:06.197 INFO:teuthology.misc:Compressing logs... 2022-07-29T09:11:06.472 INFO:teuthology.misc:Compressing logs... 2022-07-29T09:11:06.918 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/ceb924c8-0eba-11ed-8430-001a4aab830c/ceph-volume.log: file size changed while zipping 2022-07-29T09:11:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/ceb924c8-0eba-11ed-8430-001a4aab830c/ceph-volume.log: file size changed while zipping 2022-07-29T09:11:08.666 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/ceb924c8-0eba-11ed-8430-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-07-29T09:11:09.137 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/ceb924c8-0eba-11ed-8430-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-07-29T09:11:10.184 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/ceb924c8-0eba-11ed-8430-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-07-29T09:11:11.085 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/ceb924c8-0eba-11ed-8430-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-07-29T09:11:13.604 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/ceb924c8-0eba-11ed-8430-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-07-29T09:11:14.504 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/ceb924c8-0eba-11ed-8430-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-07-29T09:11:18.136 INFO:teuthology.kill:No teuthology processes running 2022-07-29T09:11:19.428 INFO:teuthology.kill:Nuking machines: ['smithi037', 'smithi138'] 2022-07-29T09:11:20.517 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_5f965ce157b8dcdaf2da92a5b5d1a63e787efdd9/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-29T09:11:20.518 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-29T09:11:20.519 INFO:teuthology.kill:2022-07-29 09:11:20,517.517 INFO:teuthology.nuke:targets: 2022-07-29T09:11:20.519 INFO:teuthology.kill: smithi037.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfUWCBowqspcJ9M/buNNfvgr5hz1o+8hNb1EXAHJ6E8mHf+FUVWG0yeYkUge4gPdT4MkJ5uBcjJvH6vr9nuZy8LtmUcWn8XZexUFl6wtBkG9YRq6mCbfyoyrZem8ZM68FOt/N8/psvliiAiRgGyllWmF/TgV45M61aFp8IN7mUEY0H8P7UD62fbNX8C/jBQH8u+sOJw16s02RPTOjOd4atdmCBZmPCFklTJO0Gteqpt67NQEjsWu5aXHKguC4jjt4eitxsfPlbMqgxY+S37wTiOKi3JeAHUh5th85lI210zV66d3swPTJuDZz2sG7NR8fp56TeAgxGqjLfM7jm7Dsd2LVDUm87w3KyBT6ZA1rmYp7vCM6ToVZShQ+3rXPJ7g4auzmF8T8sgS/hvpLd01Xlh+X3G3GBE4okYQTX96tNKHHEYXg4Ql7OnHYvZi2s+mRi03r6Y0RF8Dsg+Iz2i/fKgWiWA0V3/CtpciqN7dGFaZWOce5K4smq6osXjqesZwc= 2022-07-29T09:11:20.519 INFO:teuthology.kill: smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDczpLCF9nkRG1FdF5ReKgCSQbE0J85DEESZ1P/HFe3igCaEGI+fX2JdB5ldg4eGNigjOsYQVOpXP2U2yG8Tq4TOQ+y0IKNVO0W1JVan4eRDLkRtb2zDKS21eJwTKGPIkDRUoWCFFe7Vv4eRhHinPLD3Ff6LsSw1NUzGbcnJ2gwB7fPmSKxNrIQ5FPFBeAAD2mGIgePdaJ3TE97gL82RnaX4j/cq1pdmPNQzjdI27axAtuXqnwPdoykPKatX23tIw19ZgO5T2eH+Q6yp8qcYjdy/yVCdn+0Mk2EJJaeaI1HIUjovyWTYJIUSJYPKdJO4nQWhqiS310W57H0+6wFWEvzXy1FCCa2/3z9IaQLDJ0+SMjgoRonpYGYzJ+7ad6+xbpxF4YtXF8hlqRFfDXYMUrfE1ioJQy7R8jv2gQVz2GOjUx9jSeI1SHkZv3rxUJJz5dHK0DKW++MxbYVmencKs4O4fcLuNyTADIw4yv+aDApjzuIyMrbQ0TE0xRxMddrru8= 2022-07-29T09:11:20.520 INFO:teuthology.kill:2022-07-29 09:11:20,517.517 INFO:teuthology.nuke:Checking targets against current locks 2022-07-29T09:11:20.566 INFO:teuthology.kill:2022-07-29 09:11:20,566.566 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-29T09:11:20.596 INFO:teuthology.kill:2022-07-29 09:11:20,595.595 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-29T09:11:20.650 INFO:teuthology.kill:2022-07-29 09:11:20,650.650 INFO:teuthology.orchestra.console:Power off smithi037 2022-07-29T09:11:20.685 INFO:teuthology.kill:2022-07-29 09:11:20,685.685 INFO:teuthology.orchestra.console:Power off smithi138 2022-07-29T09:11:33.127 INFO:teuthology.kill:2022-07-29 09:11:33,127.127 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2022-07-29T09:11:33.157 INFO:teuthology.kill:2022-07-29 09:11:33,156.156 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-07-29T09:11:33.284 INFO:teuthology.kill:2022-07-29 09:11:33,283.283 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com 2022-07-29T09:11:33.304 INFO:teuthology.kill:2022-07-29 09:11:33,304.304 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com 2022-07-29T09:13:33.620 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15