2022-02-13T04:53:38.250 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-13T04:53:38.403 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging
2022-02-13T04:53:38.404 INFO:teuthology.lock.ops:Updating [ubuntu@smithi001.front.sepia.ceph.com]: reset os type and version on server
2022-02-13T04:53:38.405 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server
2022-02-13T04:53:38.423 INFO:teuthology.lock.ops:Node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging is complete
2022-02-13T04:53:38.423 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging
2022-02-13T04:53:38.424 INFO:teuthology.lock.ops:Updating [ubuntu@smithi170.front.sepia.ceph.com]: reset os type and version on server
2022-02-13T04:53:38.424 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2022-02-13T04:53:38.441 INFO:teuthology.lock.ops:Node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging is complete
2022-02-13T04:53:38.647 INFO:teuthology.provision.fog.smithi001:Scheduling deploy of centos 8.stream
2022-02-13T04:53:38.650 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 8.stream
2022-02-13T04:53:39.059 INFO:teuthology.orchestra.console:Power off smithi170
2022-02-13T04:53:39.080 INFO:teuthology.orchestra.console:Power off smithi001
2022-02-13T04:53:47.437 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2022-02-13T04:53:47.444 INFO:teuthology.orchestra.console:Power off for smithi001 completed
2022-02-13T04:53:47.539 INFO:teuthology.orchestra.console:Power on smithi170
2022-02-13T04:53:47.548 INFO:teuthology.orchestra.console:Power on smithi001
2022-02-13T04:53:55.909 INFO:teuthology.orchestra.console:Power on for smithi170 completed
2022-02-13T04:53:55.913 INFO:teuthology.orchestra.console:Power on for smithi001 completed
2022-02-13T04:53:56.010 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish
2022-02-13T04:53:56.014 INFO:teuthology.provision.fog.smithi001:Waiting for deploy to finish
2022-02-13T04:57:56.712 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-13T04:58:04.962 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-13T04:58:05.201 INFO:teuthology.provision.fog.smithi001:Node is ready
2022-02-13T04:58:05.220 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001.front.sepia.ceph.com
2022-02-13T04:58:05.279 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1 smithi001.front.sepia.ceph.com smithi001
2022-02-13T04:58:05.624 INFO:teuthology.provision.fog.smithi001:Deploy complete!
2022-02-13T04:58:21.740 INFO:teuthology.provision.fog.smithi170:Node is ready
2022-02-13T04:58:21.762 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com
2022-02-13T04:58:21.822 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170
2022-02-13T04:58:22.197 INFO:teuthology.provision.fog.smithi170:Deploy complete!
2022-02-13T04:58:22.368 INFO:teuthology.lock.ops:Checking smithi001.front.sepia.ceph.com
2022-02-13T04:58:22.369 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-13T04:58:22.389 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com
2022-02-13T04:58:22.389 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-13T04:58:22.409 INFO:teuthology.lock.ops:Updating [smithi001.front.sepia.ceph.com]: set os type and version on server
2022-02-13T04:58:22.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com'
2022-02-13T04:58:22.667 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64
2022-02-13T04:58:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream"
2022-02-13T04:58:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="8"
2022-02-13T04:58:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos"
2022-02-13T04:58:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora"
2022-02-13T04:58:22.727 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="8"
2022-02-13T04:58:22.727 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el8"
2022-02-13T04:58:22.727 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-13T04:58:22.727 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31"
2022-02-13T04:58:22.728 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-13T04:58:22.728 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/"
2022-02-13T04:58:22.728 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-13T04:58:22.728 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-13T04:58:22.728 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-13T04:58:22.730 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server
2022-02-13T04:58:22.747 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server
2022-02-13T04:58:22.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2022-02-13T04:58:22.942 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64
2022-02-13T04:58:22.999 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream"
2022-02-13T04:58:22.999 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8"
2022-02-13T04:58:22.999 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos"
2022-02-13T04:58:23.000 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora"
2022-02-13T04:58:23.000 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8"
2022-02-13T04:58:23.000 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8"
2022-02-13T04:58:23.000 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-13T04:58:23.000 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31"
2022-02-13T04:58:23.001 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-13T04:58:23.001 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/"
2022-02-13T04:58:23.001 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-13T04:58:23.001 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-13T04:58:23.002 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-13T04:58:23.002 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2022-02-13T04:58:23.097 INFO:teuthology.dispatcher.supervisor:Running job 6678994
2022-02-13T04:58:23.098 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6678994 --name teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6678994/orig.config.yaml
2022-02-13T04:58:23.101 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6678994
2022-02-13T04:58:23.101 INFO:teuthology.dispatcher.supervisor:Job PID: 6127
2022-02-13T04:58:23.102 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-13T11:28:52.431 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-13T11:28:52.630 INFO:teuthology.kill:Killing Pids: {6127}
2022-02-13T11:28:53.844 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-02-13T11:28:53.845 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-02-13T11:28:53.845 INFO:teuthology.misc:Compressing logs...
2022-02-13T11:28:53.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com'
2022-02-13T11:28:54.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2022-02-13T11:28:54.086 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-13T11:28:54.272 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-13T11:28:54.318 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-13T11:28:54.319 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now
2022-02-13T11:28:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-13T11:28:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now
2022-02-13T11:28:54.357 INFO:teuthology.misc:Compressing logs...
2022-02-13T11:28:54.788 INFO:teuthology.misc:Compressing logs...
2022-02-13T11:29:00.079 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-mgr.smithi001.mgrdch.log: file size changed while zipping
2022-02-13T11:29:20.455 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-13T11:29:23.593 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-13T11:29:40.252 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-13T11:29:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-13T11:29:53.187 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-mon.smithi001.log: file size changed while zipping
2022-02-13T11:30:12.346 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-13T11:30:15.870 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-mon.smithi170.log: file size changed while zipping
2022-02-13T11:30:20.411 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-13T11:30:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-13T11:30:47.280 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/8c1143c4-8c8a-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-13T11:31:10.173 INFO:teuthology.orchestra.run.smithi001.stderr:tar: .: file changed as we read it
2022-02-13T11:31:30.932 INFO:teuthology.kill:No teuthology processes running
2022-02-13T11:31:32.122 INFO:teuthology.kill:Nuking machines: ['smithi001', 'smithi170']
2022-02-13T11:31:33.097 INFO:teuthology.kill:2022-02-13 11:31:33,097.097 INFO:teuthology.nuke:targets:
2022-02-13T11:31:33.098 INFO:teuthology.kill:  smithi001.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3wEvingquPAUnGhP3kNErA9LYeH6/jouk3l6uPU2LqkKNt5MPL/DBbdWnser+EeYzA3Od6+0Guu/eMb3f3FLk9vQIgxowfhz1t31vrzzsSsEyWE3jISvcoNpLmsIaPTfEIZ4mxn7V4NJs0K4SIgoAx9goPdEPubgVBm7zCo52/mx1zztwigPIjVGzO2tQBLvR+f9cBXyfw5bQ6mQFFlWepvPX8wa9TSZSmn3qPjM+lJWZgouZx1bSExSWjTwuq5JnJDm3hzpMuEpcPG9cKPnOLKSSM09nqWRryyX1F2/nhmI4k9/9I3Qx/d4sr4TCOUWE46/6AQ8Er92a7PAmTOEdJBVZ/Z9NYiDB9rLMOm9nH7Da9CZsdRBhpszOelTNXGgyrBbl36LhBxDdRPF0L0i/q3MRLTwUleotNCQhXf7llA22l3kqGkkey7dtxPdqcTVLmV7u8hL4dwevN//D5WXvt+ap0Jsn7tlj/UObR7uso1J/z7MN0wSSR604pC5x0Us=
2022-02-13T11:31:33.098 INFO:teuthology.kill:  smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXG+uXgnjm4B+l6EkrFMDK9lvHfZF5vnYWldiNmYp4Vfn4iLU48iZBisq8071Fambof67RLJkUi09MOYFCJIMH9cpuUcD/KZcm67n15uNf7Pah9pzCeyxS2d5IGzj/TDc2gweck4taG/JkXPgiNqUyLDltrA92snr2bjblzfrsgGatkBC/RmxeGUlmtbbspZyVllI3iMxva541kgclJmaEXjSdAIO6tJP6Oqe8k3ZVKU/1gHlVXaW7321H8CCr9AFlXR5Zc5hjUUrJ9dfYcEMxZSSIJWzsLY6Ri0CZkJ9pLbOQAemtu9cXZW3yGOyCcpVDYKgNu9Enw3sAO7I3jGl4E/oTE81dzQLI2FYHcapVC6bnggRkz9F0V+q/I/Uk+Utx7mCu1EMAbxPOZovpHHCbC+xDeC2aySVn/yi8DWAnZAKT0iVL5M5+J2ioDNBIaluOMdYOZcFFTRhe2LlQe5C9n9QS4b78QRNmAFbWd2wz9vntZ1qwAS4nQmd3y/3x4Bs=
2022-02-13T11:31:33.125 INFO:teuthology.kill:2022-02-13 11:31:33,125.125 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-13T11:31:33.129 INFO:teuthology.kill:2022-02-13 11:31:33,128.128 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-13T11:31:33.208 INFO:teuthology.kill:2022-02-13 11:31:33,208.208 INFO:teuthology.orchestra.console:Power off smithi170
2022-02-13T11:31:33.224 INFO:teuthology.kill:2022-02-13 11:31:33,224.224 INFO:teuthology.orchestra.console:Power off smithi001
2022-02-13T11:31:41.585 INFO:teuthology.kill:2022-02-13 11:31:41,584.584 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2022-02-13T11:31:41.736 INFO:teuthology.kill:2022-02-13 11:31:41,735.735 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com
2022-02-13T11:31:45.700 INFO:teuthology.kill:2022-02-13 11:31:45,699.699 INFO:teuthology.orchestra.console:Power off for smithi001 completed
2022-02-13T11:31:45.855 INFO:teuthology.kill:2022-02-13 11:31:45,855.855 INFO:teuthology.lock.ops:unlocked: smithi001.front.sepia.ceph.com
2022-02-13T11:33:46.194 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-13T11:33:46.293 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi001 but it was locked by another job: /home/teuthworker/archive/ozeneva-2022-02-13_08:15:17-rados-wip-omri-tracing-compiled-distro-basic-smithi/6680160
2022-02-13T11:33:46.293 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi170 but it was locked by another job: /home/teuthworker/archive/ozeneva-2022-02-13_08:15:17-rados-wip-omri-tracing-compiled-distro-basic-smithi/6680161