2022-04-01T08:56:52.244 INFO:root:teuthology version: 1.1.0-d877b023
2022-04-01T08:56:52.372 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging
2022-04-01T08:56:52.373 INFO:teuthology.lock.ops:Updating [ubuntu@smithi134.front.sepia.ceph.com]: reset os type and version on server
2022-04-01T08:56:52.373 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server
2022-04-01T08:56:52.429 INFO:teuthology.lock.ops:Node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging is complete
2022-04-01T08:56:52.431 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging
2022-04-01T08:56:52.431 INFO:teuthology.lock.ops:Updating [ubuntu@smithi196.front.sepia.ceph.com]: reset os type and version on server
2022-04-01T08:56:52.431 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server
2022-04-01T08:56:52.474 INFO:teuthology.lock.ops:Node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging is complete
2022-04-01T08:56:52.652 INFO:teuthology.provision.fog.smithi134:Scheduling deploy of centos 8.stream
2022-04-01T08:56:52.676 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of centos 8.stream
2022-04-01T08:56:53.029 INFO:teuthology.orchestra.console:Power off smithi134
2022-04-01T08:56:53.066 INFO:teuthology.orchestra.console:Power off smithi196
2022-04-01T08:57:01.411 INFO:teuthology.orchestra.console:Power off for smithi134 completed
2022-04-01T08:57:01.461 INFO:teuthology.orchestra.console:Power off for smithi196 completed
2022-04-01T08:57:01.513 INFO:teuthology.orchestra.console:Power on smithi134
2022-04-01T08:57:01.562 INFO:teuthology.orchestra.console:Power on smithi196
2022-04-01T08:57:09.887 INFO:teuthology.orchestra.console:Power on for smithi134 completed
2022-04-01T08:57:09.927 INFO:teuthology.orchestra.console:Power on for smithi196 completed
2022-04-01T08:57:09.989 INFO:teuthology.provision.fog.smithi134:Waiting for deploy to finish
2022-04-01T08:57:10.028 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish
2022-04-01T09:01:09.713 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-01T09:01:35.530 INFO:teuthology.provision.fog.smithi134:Node is ready
2022-04-01T09:01:35.554 INFO:teuthology.orchestra.run.smithi134.stdout:smithi134.front.sepia.ceph.com
2022-04-01T09:01:35.613 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.15.134 smithi134.front.sepia.ceph.com smithi134
2022-04-01T09:01:35.983 INFO:teuthology.provision.fog.smithi134:Deploy complete!
2022-04-01T09:01:51.334 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-01T09:01:51.601 INFO:teuthology.provision.fog.smithi196:Node is ready
2022-04-01T09:01:51.622 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com
2022-04-01T09:01:51.681 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196
2022-04-01T09:01:52.025 INFO:teuthology.provision.fog.smithi196:Deploy complete!
2022-04-01T09:01:52.218 INFO:teuthology.lock.ops:Checking smithi134.front.sepia.ceph.com
2022-04-01T09:01:52.219 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-01T09:01:52.238 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com
2022-04-01T09:01:52.238 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-01T09:01:52.258 INFO:teuthology.lock.ops:Updating [smithi134.front.sepia.ceph.com]: set os type and version on server
2022-04-01T09:01:52.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com'
2022-04-01T09:01:52.499 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64
2022-04-01T09:01:52.567 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream"
2022-04-01T09:01:52.568 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8"
2022-04-01T09:01:52.568 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos"
2022-04-01T09:01:52.568 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora"
2022-04-01T09:01:52.569 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8"
2022-04-01T09:01:52.569 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8"
2022-04-01T09:01:52.569 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-01T09:01:52.569 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31"
2022-04-01T09:01:52.570 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-01T09:01:52.570 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/"
2022-04-01T09:01:52.570 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-01T09:01:52.570 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-01T09:01:52.571 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-01T09:01:52.572 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server
2022-04-01T09:01:52.592 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server
2022-04-01T09:01:52.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com'
2022-04-01T09:01:52.783 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64
2022-04-01T09:01:52.840 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream"
2022-04-01T09:01:52.840 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8"
2022-04-01T09:01:52.840 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos"
2022-04-01T09:01:52.841 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora"
2022-04-01T09:01:52.841 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8"
2022-04-01T09:01:52.841 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8"
2022-04-01T09:01:52.841 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-01T09:01:52.842 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31"
2022-04-01T09:01:52.842 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-01T09:01:52.842 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/"
2022-04-01T09:01:52.842 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-01T09:01:52.843 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-01T09:01:52.843 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-01T09:01:52.843 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server
2022-04-01T09:01:52.966 INFO:teuthology.dispatcher.supervisor:Running job 6771073
2022-04-01T09:01:52.966 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-04-01_01:23:52-rados-wip-yuri2-testing-2022-03-31-1523-pacific-distro-default-smithi/6771073 --name yuriw-2022-04-01_01:23:52-rados-wip-yuri2-testing-2022-03-31-1523-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-04-01_01:23:52-rados-wip-yuri2-testing-2022-03-31-1523-pacific-distro-default-smithi/6771073/orig.config.yaml
2022-04-01T09:01:52.970 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-04-01_01:23:52-rados-wip-yuri2-testing-2022-03-31-1523-pacific-distro-default-smithi/6771073
2022-04-01T09:01:52.971 INFO:teuthology.dispatcher.supervisor:Job PID: 25027
2022-04-01T09:01:52.971 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-01T15:32:21.437 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-04-01T15:32:21.586 INFO:teuthology.kill:Killing Pids: {25027}
2022-04-01T15:32:22.742 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-04-01T15:32:22.742 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-04-01T15:32:22.743 INFO:teuthology.misc:Compressing logs...
2022-04-01T15:32:22.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com'
2022-04-01T15:32:22.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com'
2022-04-01T15:32:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-04-01T15:32:23.172 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-04-01T15:32:23.229 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-04-01T15:32:23.229 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now
2022-04-01T15:32:23.282 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-04-01T15:32:23.283 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now
2022-04-01T15:32:23.284 INFO:teuthology.misc:Compressing logs...
2022-04-01T15:32:23.572 INFO:teuthology.misc:Compressing logs...
2022-04-01T15:32:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-04-01T15:32:40.138 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-04-01T15:32:50.123 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-04-01T15:32:59.184 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-04-01T15:33:01.542 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-mon.smithi134.log: file size changed while zipping
2022-04-01T15:33:02.956 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-mon.smithi196.log: file size changed while zipping
2022-04-01T15:33:16.771 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-04-01T15:33:19.735 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-04-01T15:33:22.186 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-mgr.smithi134.xqffkz.log: file size changed while zipping
2022-04-01T15:33:32.607 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-04-01T15:33:39.216 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/a9244d68-b19b-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-04-01T15:34:08.852 INFO:teuthology.orchestra.run.smithi196.stderr:tar: .: file changed as we read it
2022-04-01T15:34:08.921 INFO:teuthology.kill:No teuthology processes running
2022-04-01T15:34:10.079 INFO:teuthology.kill:Nuking machines: ['smithi134', 'smithi196']
2022-04-01T15:34:11.126 INFO:teuthology.kill:2022-04-01 15:34:11,126.126 INFO:teuthology.nuke:targets:
2022-04-01T15:34:11.127 INFO:teuthology.kill:  smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+wptqyS28ST6I4XnOojf2hR1WMf0j4pTkiEdtPH4Wme4zIqhP5N893qeIxKpXb7IhX3HHAwY2Z7cF1DRK1gR058NqCjy2AyOgaD2s/elU3dUrlDIIrM0zhgrtFVpotIjmOggEZSQJ0MQxis+/F2iWFU56LHTGpPvLA04AFGS9xAuqiQkU0bzBfFc+uzz0yTctTZCmmJ4prTW2k4ul1va0JiM7Gzwrj7cd9zd+UycUUryn7QHZow0itYzl52BQGVTKNmJvOYpcIzB8eplUKbRO92maSej3eKE56h30ozoyUZzKyZjV5ivQ79r8KIBUo4WXrTARA9RVOZ744mv3e7fhPxg3JdqJaoIFWTbvt+85bQy+/6SvZpYJS1Lza90LNHpGkxXn3GaIoy6sX/J4hcpntfhBi2B/PRNfYGC2dIBcqirwhCbj/MSuuz+o62qEAhGPHllYM9KADFCIEQHp0nMZdB1Y4fdcKrqHFljZ/M9MfW7ab/9HQq3V+HlPFQGXyBM=
2022-04-01T15:34:11.127 INFO:teuthology.kill:  smithi196.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwviGOTm1KDf4MUUhU5SbHcTaC0aw30o8WNp6B5bKuuOYtHIr/EA9eP5I9XVDUUhFHPU5zWcLB8KMwuAqttLPmIMA3h/Mbt85Bkw2MfjTyjR7Xnpbr0S3kLRE6/o9klxeJnUwLls4DYi7zyRTlijY9wBaJ+EM8v3Ovqd0ejWVIWDBFdR5x8KWFZ/tlzqZrTd2d+XqGDYtoBGdgZq3MCaZKXpr5dUcMGdkTJHC7mYeCJhr+kIG7nH3NapXVRt1JFjeVD1mXc2z1ICJWmrFHT8HQFf2d7/YRjXwePKL8z6vuBe6UQoFoFArZKWsNNyyFL4xN82M7BAUHgmJMJ/uUiOUdGZhvMPDsLR5Igsg1LRRrA+HqxA54MfPFALdkVrQOtU7S8Rhmy1WUtmkAP1AaBFyMQI46TiDuFLIzBeAw2Jo8eFAXrg4CLzB7bWrVp4QexywlraYMTxNE0qHN8LntUzVxWwN5Pov6V7ZflG6mF5aivk1CnRDdax2UoodjmXaIlYE=
2022-04-01T15:34:11.153 INFO:teuthology.kill:2022-04-01 15:34:11,153.153 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-01T15:34:11.156 INFO:teuthology.kill:2022-04-01 15:34:11,155.155 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-01T15:34:11.229 INFO:teuthology.kill:2022-04-01 15:34:11,229.229 INFO:teuthology.orchestra.console:Power off smithi134
2022-04-01T15:34:11.245 INFO:teuthology.kill:2022-04-01 15:34:11,244.244 INFO:teuthology.orchestra.console:Power off smithi196
2022-04-01T15:34:19.605 INFO:teuthology.kill:2022-04-01 15:34:19,605.605 INFO:teuthology.orchestra.console:Power off for smithi134 completed
2022-04-01T15:34:19.611 INFO:teuthology.kill:2022-04-01 15:34:19,611.611 INFO:teuthology.orchestra.console:Power off for smithi196 completed
2022-04-01T15:34:19.749 INFO:teuthology.kill:2022-04-01 15:34:19,748.748 INFO:teuthology.lock.ops:unlocked: smithi134.front.sepia.ceph.com
2022-04-01T15:34:19.752 INFO:teuthology.kill:2022-04-01 15:34:19,752.752 INFO:teuthology.lock.ops:unlocked: smithi196.front.sepia.ceph.com
2022-04-01T15:36:20.055 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-01T15:36:20.148 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi134 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-01_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi/6772520
2022-04-01T15:36:20.148 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi196 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-01_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi/6772520