2022-01-15T12:10:04.640 INFO:root:teuthology version: 1.1.0-3ff86a5d 2022-01-15T12:10:04.763 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging 2022-01-15T12:10:04.764 INFO:teuthology.lock.ops:Updating [ubuntu@smithi162.front.sepia.ceph.com]: reset os type and version on server 2022-01-15T12:10:04.764 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2022-01-15T12:10:04.782 INFO:teuthology.lock.ops:Node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging is complete 2022-01-15T12:10:04.782 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging 2022-01-15T12:10:04.783 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server 2022-01-15T12:10:04.783 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-01-15T12:10:04.802 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete 2022-01-15T12:10:04.992 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of centos 8.stream 2022-01-15T12:10:05.009 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of centos 8.stream 2022-01-15T12:10:05.306 INFO:teuthology.orchestra.console:Power off smithi162 2022-01-15T12:10:05.321 INFO:teuthology.orchestra.console:Power off smithi185 2022-01-15T12:10:13.690 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-01-15T12:10:13.791 INFO:teuthology.orchestra.console:Power on smithi185 2022-01-15T12:10:17.794 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2022-01-15T12:10:17.896 INFO:teuthology.orchestra.console:Power on smithi162 2022-01-15T12:10:22.150 INFO:teuthology.orchestra.console:Power on for smithi162 completed 2022-01-15T12:10:22.158 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2022-01-15T12:10:22.252 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish 2022-01-15T12:10:22.259 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2022-01-15T12:15:27.147 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-15T12:15:27.162 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-15T12:15:27.430 INFO:teuthology.provision.fog.smithi185:Node is ready 2022-01-15T12:15:27.441 INFO:teuthology.provision.fog.smithi162:Node is ready 2022-01-15T12:15:27.447 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2022-01-15T12:15:27.457 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com 2022-01-15T12:15:27.504 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2022-01-15T12:15:27.514 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2022-01-15T12:15:27.823 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2022-01-15T12:15:27.861 INFO:teuthology.provision.fog.smithi162:Deploy complete! 2022-01-15T12:15:28.012 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com 2022-01-15T12:15:28.013 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-15T12:15:28.033 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2022-01-15T12:15:28.033 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-15T12:15:28.052 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server 2022-01-15T12:15:28.052 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2022-01-15T12:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2022-01-15T12:15:28.292 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2022-01-15T12:15:28.292 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="8" 2022-01-15T12:15:28.293 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2022-01-15T12:15:28.293 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2022-01-15T12:15:28.293 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="8" 2022-01-15T12:15:28.293 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el8" 2022-01-15T12:15:28.294 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-15T12:15:28.294 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2022-01-15T12:15:28.294 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-15T12:15:28.294 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2022-01-15T12:15:28.295 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-15T12:15:28.295 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-15T12:15:28.296 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-15T12:15:28.297 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2022-01-15T12:15:28.315 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2022-01-15T12:15:28.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-01-15T12:15:28.501 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2022-01-15T12:15:28.557 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="CentOS Stream" 2022-01-15T12:15:28.558 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8" 2022-01-15T12:15:28.558 INFO:teuthology.orchestra.run.smithi185.stdout:ID="centos" 2022-01-15T12:15:28.558 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="rhel fedora" 2022-01-15T12:15:28.558 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8" 2022-01-15T12:15:28.559 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8" 2022-01-15T12:15:28.559 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-15T12:15:28.559 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31" 2022-01-15T12:15:28.559 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-15T12:15:28.560 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://centos.org/" 2022-01-15T12:15:28.560 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-15T12:15:28.560 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-15T12:15:28.560 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-15T12:15:28.561 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-01-15T12:15:28.669 INFO:teuthology.dispatcher.supervisor:Running job 6619431 2022-01-15T12:15:28.670 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3ff86a5dbaafecdb6414fa2fea970bb44e836829/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619431 --name yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} -- /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619431/orig.config.yaml 2022-01-15T12:15:28.675 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-15_05:47:18-rados-wip-yuri8-testing-2022-01-14-1551-distro-default-smithi/6619431 2022-01-15T12:15:28.676 INFO:teuthology.dispatcher.supervisor:Job PID: 16794 2022-01-15T12:15:28.676 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-15T18:45:57.695 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-15T18:45:57.875 INFO:teuthology.kill:Killing Pids: {16794} 2022-01-15T18:45:59.026 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-15T18:45:59.027 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-15T18:45:59.027 INFO:teuthology.misc:Compressing logs... 2022-01-15T18:45:59.028 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2022-01-15T18:45:59.209 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-01-15T18:45:59.235 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-15T18:45:59.420 INFO:teuthology.orchestra.run.smithi185.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-15T18:45:59.459 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-15T18:45:59.459 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now 2022-01-15T18:45:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-15T18:45:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:tar: Error is not recoverable: exiting now 2022-01-15T18:45:59.511 INFO:teuthology.misc:Compressing logs... 2022-01-15T18:45:59.969 INFO:teuthology.misc:Compressing logs... 2022-01-15T18:46:15.478 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-15T18:46:21.384 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-15T18:46:36.348 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-15T18:46:36.631 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-15T18:46:46.742 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-15T18:46:49.901 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-mon.smithi185.log: file size changed while zipping 2022-01-15T18:46:55.515 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-15T18:47:05.060 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-mon.smithi162.log: file size changed while zipping 2022-01-15T18:47:07.608 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-15T18:47:22.483 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-15T18:47:24.665 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/0a0d02b6-75fe-11ec-8c32-001a4aab830c/ceph-mgr.smithi162.yanasu.log: file size changed while zipping 2022-01-15T18:47:42.338 INFO:teuthology.orchestra.run.smithi162.stderr:tar: .: file changed as we read it 2022-01-15T18:47:56.825 INFO:teuthology.kill:No teuthology processes running 2022-01-15T18:47:57.964 INFO:teuthology.kill:Nuking machines: ['smithi162', 'smithi185'] 2022-01-15T18:47:58.872 INFO:teuthology.kill:2022-01-15 18:47:58,872.872 INFO:teuthology.nuke:targets: 2022-01-15T18:47:58.873 INFO:teuthology.kill: smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCTJ+ertPyOKPSNPHhGEo0ncFhWWQwQtsCG7yraGJIgYN+ve9RwlOrrhEoOPVmVIKeqWWXMPaSBlkt2jnYZb+0sC9G2yksHJRZNhf3e4F6rArJ8YvFgwReD7d0tdnPICZLmVqO1LyrIWo+YP77b+x3HZvkyXYwV1NLQadxWH1BLogMobLng1WBUZFDzuAeLBQiqoT084e2aKFozuGI0Ey6b4bBcYheiaBGHp7wPbWAVBaPfENme93aQVGB4UuiwLCpT6CjhQuQpVEtly7vD/BgTUcV9X4DDm98mPeGh05dxeEUBrTJ56hX8IGRehRwpp+NeGU7proLF44mXGf6myAFV7fGJ5eVz4k6VPkOGIocy18n97R1HUFJpelKKpMBdz375F3FqUmctWQ7Z+AGFLsASHwMhT1/Sfhz1WBwOSbTFd4UCqrTb42h5Woke0ECzJEvrZghGMSDd7uvn6PgxXn9r3Wq4XZgHU0seJKQfUfXj9RWfH7zjep3MQxqfHecD4Sc= 2022-01-15T18:47:58.873 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbpLZSHVK4i9h+s5DG28rAcKVnMfiu/qnw8LEh/025/f7RWXu4LBZLHnrWFb/oVsYxt0p+ky3mvE4rrkte1SxSRPNV8gYKd0MfAqGFq1bz5msMOwEUUp2LyUHzIh59IWKuKLPD/YGZbAAuShjkHMN4OX6i1cv/qk7/EIp9yPpquvWFsq9uRgHSjank+m0oJaCaNH4KoIz0A/27sHTo9VF6X60v8o5S3OYGXb2S2jkcjGKpNgXxZsO6cDydkJMjhD/CsOtnYkDdpV8nG6xxOi81fsudGzqD8rV4A7S5ruOTvI+i/j1r1i1hmD6sVsbQZJMrfIYK4V7N2lkcAx2ZSbtoQWTG9QTrrf5j83rDoufpJzOVMkuqhhqUx+fpyekIYxiIEx0Abn0EpQ54wER+vw4VnxouuJyGQIWvufGBfQidA82rm9cWtD75vHVPMsP41TnkB/Puf7OV2AbiUKYZYT24Ecxkwmc53Q7nGwDZGvYCcOvP6UkZ0PKgh1pXhaqQR/0= 2022-01-15T18:47:58.898 INFO:teuthology.kill:2022-01-15 18:47:58,898.898 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-15T18:47:58.905 INFO:teuthology.kill:2022-01-15 18:47:58,905.905 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-15T18:47:58.968 INFO:teuthology.kill:2022-01-15 18:47:58,967.967 INFO:teuthology.orchestra.console:Power off smithi185 2022-01-15T18:47:58.980 INFO:teuthology.kill:2022-01-15 18:47:58,980.980 INFO:teuthology.orchestra.console:Power off smithi162 2022-01-15T18:48:07.346 INFO:teuthology.kill:2022-01-15 18:48:07,346.346 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2022-01-15T18:48:07.495 INFO:teuthology.kill:2022-01-15 18:48:07,494.494 INFO:teuthology.lock.ops:unlocked: smithi162.front.sepia.ceph.com 2022-01-15T18:48:11.461 INFO:teuthology.kill:2022-01-15 18:48:11,461.461 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-01-15T18:48:11.608 INFO:teuthology.kill:2022-01-15 18:48:11,608.608 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com 2022-01-15T18:50:11.918 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-15T18:50:12.010 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi162 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-15_15:16:34-upgrade-wip-yuriw-master-RC0-1.14.22-distro-default-smithi/6619905 2022-01-15T18:50:12.011 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-15_15:16:34-upgrade-wip-yuriw-master-RC0-1.14.22-distro-default-smithi/6619905