2021-12-16T11:21:08.559 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-16T11:21:08.680 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging 2021-12-16T11:21:08.681 INFO:teuthology.lock.ops:Updating [ubuntu@smithi001.front.sepia.ceph.com]: reset os type and version on server 2021-12-16T11:21:08.682 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2021-12-16T11:21:08.702 INFO:teuthology.lock.ops:Node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging is complete 2021-12-16T11:21:08.702 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging 2021-12-16T11:21:08.703 INFO:teuthology.lock.ops:Updating [ubuntu@smithi149.front.sepia.ceph.com]: reset os type and version on server 2021-12-16T11:21:08.704 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2021-12-16T11:21:08.725 INFO:teuthology.lock.ops:Node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging is complete 2021-12-16T11:21:08.914 INFO:teuthology.provision.fog.smithi001:Scheduling deploy of centos 8.stream 2021-12-16T11:21:08.929 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of centos 8.stream 2021-12-16T11:21:09.254 INFO:teuthology.orchestra.console:Power off smithi001 2021-12-16T11:21:09.273 INFO:teuthology.orchestra.console:Power off smithi149 2021-12-16T11:21:17.635 INFO:teuthology.orchestra.console:Power off for smithi001 completed 2021-12-16T11:21:17.737 INFO:teuthology.orchestra.console:Power on smithi001 2021-12-16T11:21:21.753 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2021-12-16T11:21:21.855 INFO:teuthology.orchestra.console:Power on smithi149 2021-12-16T11:21:26.100 INFO:teuthology.orchestra.console:Power on for smithi001 completed 2021-12-16T11:21:26.105 INFO:teuthology.orchestra.console:Power on for smithi149 completed 2021-12-16T11:21:26.202 INFO:teuthology.provision.fog.smithi001:Waiting for deploy to finish 2021-12-16T11:21:26.208 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish 2021-12-16T11:25:07.170 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-16T11:25:26.959 INFO:teuthology.provision.fog.smithi001:Node is ready 2021-12-16T11:25:26.981 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001.front.sepia.ceph.com 2021-12-16T11:25:27.040 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1 smithi001.front.sepia.ceph.com smithi001 2021-12-16T11:25:27.397 INFO:teuthology.provision.fog.smithi001:Deploy complete! 2021-12-16T11:26:31.471 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-16T11:26:31.764 INFO:teuthology.provision.fog.smithi149:Node is ready 2021-12-16T11:26:31.781 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com 2021-12-16T11:26:31.838 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149 2021-12-16T11:26:32.159 INFO:teuthology.provision.fog.smithi149:Deploy complete! 2021-12-16T11:26:32.340 INFO:teuthology.lock.ops:Checking smithi001.front.sepia.ceph.com 2021-12-16T11:26:32.341 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-16T11:26:32.361 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com 2021-12-16T11:26:32.361 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-16T11:26:32.380 INFO:teuthology.lock.ops:Updating [smithi001.front.sepia.ceph.com]: set os type and version on server 2021-12-16T11:26:32.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2021-12-16T11:26:32.632 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2021-12-16T11:26:32.689 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream" 2021-12-16T11:26:32.689 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="8" 2021-12-16T11:26:32.689 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2021-12-16T11:26:32.689 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2021-12-16T11:26:32.690 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="8" 2021-12-16T11:26:32.690 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el8" 2021-12-16T11:26:32.690 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-16T11:26:32.690 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2021-12-16T11:26:32.691 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-16T11:26:32.691 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/" 2021-12-16T11:26:32.691 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-16T11:26:32.691 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-16T11:26:32.692 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-16T11:26:32.693 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2021-12-16T11:26:32.711 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server 2021-12-16T11:26:32.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2021-12-16T11:26:32.898 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2021-12-16T11:26:32.955 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2021-12-16T11:26:32.955 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8" 2021-12-16T11:26:32.955 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2021-12-16T11:26:32.955 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2021-12-16T11:26:32.956 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8" 2021-12-16T11:26:32.956 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8" 2021-12-16T11:26:32.956 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-16T11:26:32.957 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2021-12-16T11:26:32.957 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-16T11:26:32.957 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2021-12-16T11:26:32.957 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-16T11:26:32.958 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-16T11:26:32.958 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-16T11:26:32.959 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2021-12-16T11:26:33.051 INFO:teuthology.dispatcher.supervisor:Running job 6566447 2021-12-16T11:26:33.051 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_swagner@teuthology --archive /home/teuthworker/archive/swagner-2021-12-16_10:25:44-orch:cephadm-wip-swagner4-testing-2021-12-15-1536-distro-default-smithi/6566447 --name swagner-2021-12-16_10:25:44-orch:cephadm-wip-swagner4-testing-2021-12-15-1536-distro-default-smithi --description orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/swagner-2021-12-16_10:25:44-orch:cephadm-wip-swagner4-testing-2021-12-15-1536-distro-default-smithi/6566447/orig.config.yaml 2021-12-16T11:26:33.055 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/swagner-2021-12-16_10:25:44-orch:cephadm-wip-swagner4-testing-2021-12-15-1536-distro-default-smithi/6566447 2021-12-16T11:26:33.056 INFO:teuthology.dispatcher.supervisor:Job PID: 16090 2021-12-16T11:26:33.056 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-16T23:26:53.812 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-16T23:26:53.997 INFO:teuthology.kill:Killing Pids: {16090} 2021-12-16T23:26:55.092 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-12-16T23:26:55.093 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-12-16T23:26:55.094 INFO:teuthology.misc:Compressing logs... 2021-12-16T23:26:55.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2021-12-16T23:26:55.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2021-12-16T23:26:55.338 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2021-12-16T23:26:55.664 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2021-12-16T23:26:55.706 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2021-12-16T23:26:55.706 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now 2021-12-16T23:26:55.749 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2021-12-16T23:26:55.749 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2021-12-16T23:26:55.759 INFO:teuthology.misc:Compressing logs... 2021-12-16T23:26:56.155 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-16T23:26:56.354 INFO:teuthology.misc:Compressing logs... 2021-12-16T23:26:56.644 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-mgr.smithi001.fgbomz.log: file size changed while zipping 2021-12-16T23:27:00.549 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-mgr.smithi149.ecipjn.log: file size changed while zipping 2021-12-16T23:27:25.289 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2021-12-16T23:27:30.081 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2021-12-16T23:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-mon.smithi001.log: file size changed while zipping 2021-12-16T23:27:55.490 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2021-12-16T23:28:13.286 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2021-12-16T23:28:20.907 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2021-12-16T23:28:31.841 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2021-12-16T23:28:42.817 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2021-12-16T23:28:52.743 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-mon.smithi149.log: file size changed while zipping 2021-12-16T23:28:59.858 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/c34badac-5e63-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2021-12-16T23:29:50.298 INFO:teuthology.kill:No teuthology processes running 2021-12-16T23:29:51.338 INFO:teuthology.kill:Nuking machines: ['smithi001', 'smithi149'] 2021-12-16T23:29:52.241 INFO:teuthology.kill:2021-12-16 23:29:52,240.240 INFO:teuthology.nuke:targets: 2021-12-16T23:29:52.241 INFO:teuthology.kill: smithi001.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDrUE6wz6F9uf1Ai/Jopr6jXebOhXmpojXUB9aiOp4lJDwtVYSrmgI7B32RibHAN1AtupYyp6Nbgt2dg8OXN6X+1C7XNmweXn8OM+ohtlkWiS+893fsYVGs9znN5tJ2ZZ0R6TxJ42ruesrou47Yw4ucMz1fDEkowdv0kPkEDZLzLHMoMBphFJVohNOIaRYljoTT6CqXU1SAsn+Wbq6HnraygpAArQxRUUR2Hv+25pDRVbJATM6JyJ4typVaJkuKNIngvOdMs89Vsj90uxpFpA83i4Zx+COnLNEn7386AnlyhH52qNLlt/ZcQVYgr702hKE0eO5VCGZAu/I/qYZ+QdH9sDLXPzsPcASJWzhM1JHlcF2o/CwfNZCQOVxt5CMgOvNyGx4TvsFxa9ifVsK+e4qfegm/bL6bVGSMwpZ/H8ispxvt27AGI7aeOXPA9H1k5zR4HYkJL60px+ic2++OlTThP+lsuqQCkmgVDHVIJkzDESMN4E0ytZLzmaJZY1uTT8M= 2021-12-16T23:29:52.242 INFO:teuthology.kill: smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu43E322zA8F4hWMFLojlYOgV46cT9aEf/gbFT5f5xNStdOdSBllo46sQLgKbKiBRHTrviGVe/3cP6poCF9aEFdqGcIjKJtlVm0msSfvQxS7If/7bitETbmqxXU4Uy4OAQGGghM7rKyoBYUVnrWe2q1HIxFp+v/mNBpOwxhIV/a5kyzQGbx6sDd2OBV6wT1W+N2tUTJOtOwx2oINaveHrUeYSP1pT9wX6sKxuBs889WSf1JgqwsGrpLx+nOmZZ6/K9Ff9tsk0HQmHytTSWHji70gfkCLo5pkz193kNJ+2sUOqgi/nUrbAzOvQIPzaQTYPdyzOWF6Snrm1UslBPmWAwZBElKrod1Wpl5oL5EYP8XKfG1T2rhw6+aI7rWeEOSXN6hzIIJiEn38DC3PEmpBsZO3PXer1ZrHReFn1LwEYrTbiSsy49ezLuTBDgU69rq0n0ja2Ao/BxiBZwsMpXTRRlmeZEcFef5MJdHxKIne8yg6Qhun8/jZSix66/aIWNBmU= 2021-12-16T23:29:52.266 INFO:teuthology.kill:2021-12-16 23:29:52,266.266 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-16T23:29:52.269 INFO:teuthology.kill:2021-12-16 23:29:52,269.269 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-16T23:29:52.344 INFO:teuthology.kill:2021-12-16 23:29:52,343.343 INFO:teuthology.orchestra.console:Power off smithi149 2021-12-16T23:29:52.356 INFO:teuthology.kill:2021-12-16 23:29:52,356.356 INFO:teuthology.orchestra.console:Power off smithi001 2021-12-16T23:30:04.829 INFO:teuthology.kill:2021-12-16 23:30:04,828.828 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2021-12-16T23:30:04.834 INFO:teuthology.kill:2021-12-16 23:30:04,834.834 INFO:teuthology.orchestra.console:Power off for smithi001 completed 2021-12-16T23:30:04.980 INFO:teuthology.kill:2021-12-16 23:30:04,979.979 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com 2021-12-16T23:30:04.984 INFO:teuthology.kill:2021-12-16 23:30:04,984.984 INFO:teuthology.lock.ops:unlocked: smithi001.front.sepia.ceph.com 2021-12-16T23:32:05.202 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15