2022-01-19T23:37:49.592 INFO:root:teuthology version: 1.1.0-3ff86a5d
2022-01-19T23:37:49.765 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging
2022-01-19T23:37:49.766 INFO:teuthology.lock.ops:Updating [ubuntu@smithi170.front.sepia.ceph.com]: reset os type and version on server
2022-01-19T23:37:49.766 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2022-01-19T23:37:49.787 INFO:teuthology.lock.ops:Node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging is complete
2022-01-19T23:37:49.787 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging
2022-01-19T23:37:49.788 INFO:teuthology.lock.ops:Updating [ubuntu@smithi196.front.sepia.ceph.com]: reset os type and version on server
2022-01-19T23:37:49.788 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server
2022-01-19T23:37:49.810 INFO:teuthology.lock.ops:Node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging is complete
2022-01-19T23:37:50.010 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 8.stream
2022-01-19T23:37:50.047 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of centos 8.stream
2022-01-19T23:37:50.419 INFO:teuthology.orchestra.console:Power off smithi170
2022-01-19T23:37:50.483 INFO:teuthology.orchestra.console:Power off smithi196
2022-01-19T23:37:58.788 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2022-01-19T23:37:58.889 INFO:teuthology.orchestra.console:Power on smithi170
2022-01-19T23:38:02.981 INFO:teuthology.orchestra.console:Power off for smithi196 completed
2022-01-19T23:38:03.083 INFO:teuthology.orchestra.console:Power on smithi196
2022-01-19T23:38:07.274 INFO:teuthology.orchestra.console:Power on for smithi170 completed
2022-01-19T23:38:07.375 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish
2022-01-19T23:38:11.537 INFO:teuthology.orchestra.console:Power on for smithi196 completed
2022-01-19T23:38:11.638 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish
2022-01-19T23:42:09.159 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-19T23:42:34.328 INFO:teuthology.provision.fog.smithi196:Node is ready
2022-01-19T23:42:34.349 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com
2022-01-19T23:42:34.406 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196
2022-01-19T23:42:34.768 INFO:teuthology.provision.fog.smithi196:Deploy complete!
2022-01-19T23:42:39.615 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-19T23:42:39.890 INFO:teuthology.provision.fog.smithi170:Node is ready
2022-01-19T23:42:39.907 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com
2022-01-19T23:42:39.965 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170
2022-01-19T23:42:40.317 INFO:teuthology.provision.fog.smithi170:Deploy complete!
2022-01-19T23:42:40.466 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com
2022-01-19T23:42:40.467 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-19T23:42:40.488 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com
2022-01-19T23:42:40.488 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-19T23:42:40.508 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server
2022-01-19T23:42:40.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2022-01-19T23:42:40.693 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64
2022-01-19T23:42:40.748 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream"
2022-01-19T23:42:40.748 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8"
2022-01-19T23:42:40.748 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos"
2022-01-19T23:42:40.749 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora"
2022-01-19T23:42:40.749 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8"
2022-01-19T23:42:40.749 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8"
2022-01-19T23:42:40.750 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-19T23:42:40.750 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31"
2022-01-19T23:42:40.750 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-19T23:42:40.750 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/"
2022-01-19T23:42:40.751 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-19T23:42:40.751 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-19T23:42:40.751 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-19T23:42:40.752 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2022-01-19T23:42:40.776 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server
2022-01-19T23:42:40.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com'
2022-01-19T23:42:40.972 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64
2022-01-19T23:42:41.030 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream"
2022-01-19T23:42:41.031 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8"
2022-01-19T23:42:41.031 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos"
2022-01-19T23:42:41.031 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora"
2022-01-19T23:42:41.031 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8"
2022-01-19T23:42:41.032 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8"
2022-01-19T23:42:41.032 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-19T23:42:41.033 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31"
2022-01-19T23:42:41.033 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-19T23:42:41.033 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/"
2022-01-19T23:42:41.033 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-19T23:42:41.033 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-19T23:42:41.034 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-19T23:42:41.035 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server
2022-01-19T23:42:41.136 INFO:teuthology.dispatcher.supervisor:Running job 6627952
2022-01-19T23:42:41.136 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3ff86a5dbaafecdb6414fa2fea970bb44e836829/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-19_19:38:53-rados-wip-yuri5-testing-2022-01-19-0916-pacific-distro-default-smithi/6627952 --name yuriw-2022-01-19_19:38:53-rados-wip-yuri5-testing-2022-01-19-0916-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-01-19_19:38:53-rados-wip-yuri5-testing-2022-01-19-0916-pacific-distro-default-smithi/6627952/orig.config.yaml
2022-01-19T23:42:41.142 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-19_19:38:53-rados-wip-yuri5-testing-2022-01-19-0916-pacific-distro-default-smithi/6627952
2022-01-19T23:42:41.143 INFO:teuthology.dispatcher.supervisor:Job PID: 1787
2022-01-19T23:42:41.143 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-20T06:13:10.320 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-01-20T06:13:10.549 INFO:teuthology.kill:Killing Pids: {1787}
2022-01-20T06:13:11.709 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-01-20T06:13:11.710 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-01-20T06:13:11.711 INFO:teuthology.misc:Compressing logs...
2022-01-20T06:13:11.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2022-01-20T06:13:11.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com'
2022-01-20T06:13:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-20T06:13:12.216 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-20T06:13:12.258 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-20T06:13:12.258 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now
2022-01-20T06:13:12.383 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-20T06:13:12.383 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now
2022-01-20T06:13:12.385 INFO:teuthology.misc:Compressing logs...
2022-01-20T06:13:12.682 INFO:teuthology.misc:Compressing logs...
2022-01-20T06:13:16.080 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-mon.smithi196.log: file size changed while zipping
2022-01-20T06:13:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-01-20T06:13:46.863 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-01-20T06:14:01.281 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-01-20T06:14:13.017 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-01-20T06:14:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-01-20T06:14:35.099 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-01-20T06:14:43.067 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-01-20T06:14:47.575 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-mgr.smithi170.loccxq.log: file size changed while zipping
2022-01-20T06:14:56.265 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-01-20T06:14:59.027 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/9a89b29e-7982-11ec-8c34-001a4aab830c/ceph-mon.smithi170.log: file size changed while zipping
2022-01-20T06:15:21.365 INFO:teuthology.orchestra.run.smithi170.stderr:tar: .: file changed as we read it
2022-01-20T06:15:42.283 INFO:teuthology.kill:No teuthology processes running
2022-01-20T06:15:43.454 INFO:teuthology.kill:Nuking machines: ['smithi170', 'smithi196']
2022-01-20T06:15:44.430 INFO:teuthology.kill:2022-01-20 06:15:44,429.429 INFO:teuthology.nuke:targets:
2022-01-20T06:15:44.431 INFO:teuthology.kill:  smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSLRwgoDHfLxaBDnEqbtVIpPIEWE7kV7nV06k/tc3p+k8UrUiLRe8S4Im0CmJUzGPtW+0oHGMRQiXkGATB3VVjdwR1CJbNqg+vibUbDlch5A4bBXYbKN4JftlB1lU983eqYLbktkyPda0CyTHQDHDdHUZEyUdfzC3a2vAoaru+++AS8q2TVq+r4S3dhB05H/RddtcpQVrOnuBu0otyyCOl1qjv/l88ftC321tpWLwgvZIcZm288LoMyxjSxiRUE+rXmyFleUjzEqnjtno3dIaiCkK6+Z3stbqje8oqmcRDUJkTuGVYeEPADH0e8U1w3ELg2S2Sv81n2T2z3nY1cJY7pNVumjePB5ANdEvVq4AR6/jqAbTpAQ8for2GVrXsTPXREH8KGhiROOXLyhPqqOifzaoko6lv8Rf3hLVncktZ/Oshqprvm3h+XfpALVKDUFjkwei7RbvdSY/SHg3oBSKEwwvTTcWJEzCSTWppqNgc2CFNZJXdG8xsCGH7QftEmv0=
2022-01-20T06:15:44.431 INFO:teuthology.kill:  smithi196.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfFwmu57IWTDZgv5wR5gxfEtp8rAm3xMuMT3CUwEUJvbGwrOgFahrN+x1pu6T851RG5WVekIgSwJkGbeV4SpP88lgU1p3J9IXfrhz7phVjVR5eIm+XNpP/mNds4L1Solr96Nknxxr9PyecRvWnaSmsgxHLSqWzuqSKyvMB+0UbDrBZSQ4EmmP6IP3/zr5HbST6WLtPtcSRp/TiXxY7132+WxzdUuEjvkzDWpE6cmXOpTl4uhiVCjrW/VC8/q4BBat+ovyYenpAkvm2Z299BCzbfFghX9aurUtg5wYC5xotkzPuIkTRXCqlbkfCKYwQMgdCAjqFvxExIWwtfsyxCjQk7nMofMpsqSbNroGPxgD61mXk0HKfXSTO8KuygjpYMZEP0ZV8weuqo5SrKscmy5sgQV3Q26W9xTCLz05FiC0eBMG1/+W90M1sYsyWYlNSYMeqRUNIkuVXKfypePHw19hOdzs0Nf52kitSSrj5Au5ynHL74+VG3Och8s3KGZTvbyE=
2022-01-20T06:15:44.456 INFO:teuthology.kill:2022-01-20 06:15:44,456.456 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-20T06:15:44.459 INFO:teuthology.kill:2022-01-20 06:15:44,459.459 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-20T06:15:44.544 INFO:teuthology.kill:2022-01-20 06:15:44,544.544 INFO:teuthology.orchestra.console:Power off smithi196
2022-01-20T06:15:44.556 INFO:teuthology.kill:2022-01-20 06:15:44,556.556 INFO:teuthology.orchestra.console:Power off smithi170
2022-01-20T06:15:52.913 INFO:teuthology.kill:2022-01-20 06:15:52,912.912 INFO:teuthology.orchestra.console:Power off for smithi196 completed
2022-01-20T06:15:52.918 INFO:teuthology.kill:2022-01-20 06:15:52,918.918 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2022-01-20T06:15:53.064 INFO:teuthology.kill:2022-01-20 06:15:53,063.063 INFO:teuthology.lock.ops:unlocked: smithi196.front.sepia.ceph.com
2022-01-20T06:15:53.067 INFO:teuthology.kill:2022-01-20 06:15:53,067.067 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com
2022-01-20T06:17:53.415 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-20T06:17:53.521 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi170 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-20_05:07:02-rgw-pacific-distro-default-smithi/6628536
2022-01-20T06:17:53.521 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi196 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-20_05:07:02-rgw-pacific-distro-default-smithi/6628536