2022-04-06T15:05:39.352 INFO:root:teuthology version: 1.1.0-3dd79763
2022-04-06T15:05:39.483 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging
2022-04-06T15:05:39.484 INFO:teuthology.lock.ops:Updating [ubuntu@smithi082.front.sepia.ceph.com]: reset os type and version on server
2022-04-06T15:05:39.485 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server
2022-04-06T15:05:39.502 INFO:teuthology.lock.ops:Node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging is complete
2022-04-06T15:05:39.502 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging
2022-04-06T15:05:39.502 INFO:teuthology.lock.ops:Updating [ubuntu@smithi150.front.sepia.ceph.com]: reset os type and version on server
2022-04-06T15:05:39.503 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server
2022-04-06T15:05:39.521 INFO:teuthology.lock.ops:Node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging is complete
2022-04-06T15:05:39.731 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of centos 8.stream
2022-04-06T15:05:39.738 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of centos 8.stream
2022-04-06T15:05:40.053 INFO:teuthology.orchestra.console:Power off smithi082
2022-04-06T15:05:40.069 INFO:teuthology.orchestra.console:Power off smithi150
2022-04-06T15:05:48.431 INFO:teuthology.orchestra.console:Power off for smithi150 completed
2022-04-06T15:05:48.533 INFO:teuthology.orchestra.console:Power on smithi150
2022-04-06T15:05:52.540 INFO:teuthology.orchestra.console:Power off for smithi082 completed
2022-04-06T15:05:52.642 INFO:teuthology.orchestra.console:Power on smithi082
2022-04-06T15:05:56.902 INFO:teuthology.orchestra.console:Power on for smithi150 completed
2022-04-06T15:05:57.004 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish
2022-04-06T15:06:01.009 INFO:teuthology.orchestra.console:Power on for smithi082 completed
2022-04-06T15:06:01.110 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish
2022-04-06T15:10:44.907 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-06T15:10:45.199 INFO:teuthology.provision.fog.smithi082:Node is ready
2022-04-06T15:10:45.217 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082.front.sepia.ceph.com
2022-04-06T15:10:45.274 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82 smithi082.front.sepia.ceph.com smithi082
2022-04-06T15:10:45.617 INFO:teuthology.provision.fog.smithi082:Deploy complete!
2022-04-06T15:10:50.109 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-06T15:10:50.397 INFO:teuthology.provision.fog.smithi150:Node is ready
2022-04-06T15:10:50.415 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com
2022-04-06T15:10:50.473 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150
2022-04-06T15:10:50.827 INFO:teuthology.provision.fog.smithi150:Deploy complete!
2022-04-06T15:10:51.019 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com
2022-04-06T15:10:51.020 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-06T15:10:51.041 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com
2022-04-06T15:10:51.041 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-06T15:10:51.058 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server
2022-04-06T15:10:51.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com'
2022-04-06T15:10:51.253 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64
2022-04-06T15:10:51.310 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream"
2022-04-06T15:10:51.311 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8"
2022-04-06T15:10:51.311 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos"
2022-04-06T15:10:51.311 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora"
2022-04-06T15:10:51.311 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8"
2022-04-06T15:10:51.312 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8"
2022-04-06T15:10:51.312 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-06T15:10:51.312 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31"
2022-04-06T15:10:51.312 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-06T15:10:51.313 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/"
2022-04-06T15:10:51.313 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-06T15:10:51.313 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-06T15:10:51.313 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-06T15:10:51.315 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server
2022-04-06T15:10:51.332 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server
2022-04-06T15:10:51.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com'
2022-04-06T15:10:51.525 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64
2022-04-06T15:10:51.581 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream"
2022-04-06T15:10:51.581 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8"
2022-04-06T15:10:51.582 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos"
2022-04-06T15:10:51.582 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora"
2022-04-06T15:10:51.582 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8"
2022-04-06T15:10:51.582 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8"
2022-04-06T15:10:51.583 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-06T15:10:51.583 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31"
2022-04-06T15:10:51.583 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-06T15:10:51.583 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/"
2022-04-06T15:10:51.583 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-06T15:10:51.584 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-06T15:10:51.584 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-06T15:10:51.584 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server
2022-04-06T15:10:51.682 INFO:teuthology.dispatcher.supervisor:Running job 6779332
2022-04-06T15:10:51.682 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3dd797633f37e2d7e2c4ae3ef9ac35a188c2028d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-04-06_14:02:46-rados-wip-yuri4-testing-2022-04-05-1720-pacific-distro-default-smithi/6779332 --name yuriw-2022-04-06_14:02:46-rados-wip-yuri4-testing-2022-04-05-1720-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-04-06_14:02:46-rados-wip-yuri4-testing-2022-04-05-1720-pacific-distro-default-smithi/6779332/orig.config.yaml
2022-04-06T15:10:51.686 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-04-06_14:02:46-rados-wip-yuri4-testing-2022-04-05-1720-pacific-distro-default-smithi/6779332
2022-04-06T15:10:51.687 INFO:teuthology.dispatcher.supervisor:Job PID: 5463
2022-04-06T15:10:51.687 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-06T21:41:20.588 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-04-06T21:41:20.735 INFO:teuthology.kill:Killing Pids: {5463}
2022-04-06T21:41:21.943 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-04-06T21:41:21.944 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-04-06T21:41:21.945 INFO:teuthology.misc:Compressing logs...
2022-04-06T21:41:21.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com'
2022-04-06T21:41:22.139 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com'
2022-04-06T21:41:22.188 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-04-06T21:41:22.378 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-04-06T21:41:22.422 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-04-06T21:41:22.422 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now
2022-04-06T21:41:22.465 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-04-06T21:41:22.465 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now
2022-04-06T21:41:22.466 INFO:teuthology.misc:Compressing logs...
2022-04-06T21:41:22.721 INFO:teuthology.misc:Compressing logs...
2022-04-06T21:41:40.083 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-04-06T21:41:41.944 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-04-06T21:41:54.641 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-04-06T21:42:00.120 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-04-06T21:42:03.880 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-mon.smithi150.log: file size changed while zipping
2022-04-06T21:42:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-mon.smithi082.log: file size changed while zipping
2022-04-06T21:42:20.413 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-04-06T21:42:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-04-06T21:42:35.363 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-04-06T21:42:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-04-06T21:42:42.679 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/60b86d14-b5bc-11ec-8c36-001a4aab830c/ceph-mgr.smithi082.xdyjqv.log: file size changed while zipping
2022-04-06T21:43:14.456 INFO:teuthology.kill:No teuthology processes running
2022-04-06T21:43:15.621 INFO:teuthology.kill:Nuking machines: ['smithi082', 'smithi150']
2022-04-06T21:43:16.600 INFO:teuthology.kill:2022-04-06 21:43:16,600.600 INFO:teuthology.nuke:targets:
2022-04-06T21:43:16.602 INFO:teuthology.kill:  smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjCaUkDAwI7FKkC3+pdsK0LLdg7cKInlfiguxR122RvhIg7erQKcpD+zAGu2mm/8scLx0M5K2T9rdsFB7jOF/Rv8XWR5BqoVLBbkLw/I8dUGcg/gbf2dXUNG7WVKDGZhGtKhiZ8SiwII3bh+u53woDXjcNP7DW/gg185eHYswWkdYBdQeebSOk+Bf2EFBs1OiHjREcAye+JLuXnmaCmeLQ3GVvrIYDJF2hneg635pM8CNyHGCBvMLJs2uy2sP0SBj0K4Pns7cyEUFUngmJlXlGDGYQ+JI/DmkxlLOsbKC92kxhaQYTJcjRjfRouj4YYotWc+ZDtTvK9CPSngYiFUiu5lFJ5lxJqP4Pf8j9H3J8HZi251p2z3NaT89SZkNVNrpdsSkxNZUTnQM2UmOaIzdYwsBSv6OoXh6Ca0cDMFPMKXcmQilViY4kZ8v9aq2zydqq65buCAC40aBjbEVaei690WLu8RY15bviVKxI5uq7qsjl/sErtmnzApof/T6AFKU=
2022-04-06T21:43:16.602 INFO:teuthology.kill:  smithi150.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWp+9wXtXTcIIoeADZk0y701Iv9/YfrrPA0bFfYtWgF8kBEregrX+K59bG6dD2P3nCt/3zHksp2LZfaRgZn6nbcdrl0rsAWC7ZyY3PyNaDO6eXARx8ctLnp2bg2/1BeJp58vIkX5+dQEItJQ5xxy0ys0MdSwt5vvtVB08gS8Whomm6pmMy2JPnOWcg9BbLDA9sjep6B++rZzny6KadEvpKRDO8Li6T9AcwxiQA8H8N8c91ofPFMfsPSZ0ktXer6s5Gaqj/XzMZOLqhc9xSm3TFMYp05avuXrAQdbcALi3OoKZL688cI7djEZbiPHi7LxQ81QkSmQLjC+jkba7mJVXxR3syssFkVBX6yT2lJ4QlvFHrHZq1k2k0NVH1RDM6c8VvmLfy8pWJZD3GAcoIwt16vUwzbD3SNiTh+j5eDSnLjJw7Zevl9+Q2rcvol/oBMZbaDa/RbENAOmDb3DYvwOtt0IPITfSV6FMat9wgidu5wlU3mCo2lVsI3NDgRVSWpZM=
2022-04-06T21:43:16.627 INFO:teuthology.kill:2022-04-06 21:43:16,627.627 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-06T21:43:16.630 INFO:teuthology.kill:2022-04-06 21:43:16,630.630 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-06T21:43:16.706 INFO:teuthology.kill:2022-04-06 21:43:16,705.705 INFO:teuthology.orchestra.console:Power off smithi082
2022-04-06T21:43:16.725 INFO:teuthology.kill:2022-04-06 21:43:16,724.724 INFO:teuthology.orchestra.console:Power off smithi150
2022-04-06T21:43:25.144 INFO:teuthology.kill:2022-04-06 21:43:25,143.143 INFO:teuthology.orchestra.console:Power off for smithi150 completed
2022-04-06T21:43:25.149 INFO:teuthology.kill:2022-04-06 21:43:25,149.149 INFO:teuthology.orchestra.console:Power off for smithi082 completed
2022-04-06T21:43:25.292 INFO:teuthology.kill:2022-04-06 21:43:25,291.291 INFO:teuthology.lock.ops:unlocked: smithi150.front.sepia.ceph.com
2022-04-06T21:43:25.296 INFO:teuthology.kill:2022-04-06 21:43:25,295.295 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com
2022-04-06T21:45:25.615 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-06T21:45:25.712 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi082 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-04-06_16:35:43-rados-wip-yuri5-testing-2022-04-05-1720-distro-default-smithi/6780078
2022-04-06T21:45:25.712 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi150 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-04-06_16:35:43-rados-wip-yuri5-testing-2022-04-05-1720-distro-default-smithi/6780078