2021-12-19T16:37:35.533 INFO:root:teuthology version: 1.1.0-95a7d479
2021-12-19T16:37:35.660 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi090.front.sepia.ceph.com' reimaging
2021-12-19T16:37:35.661 INFO:teuthology.lock.ops:Updating [ubuntu@smithi090.front.sepia.ceph.com]: reset os type and version on server
2021-12-19T16:37:35.662 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2021-12-19T16:37:35.681 INFO:teuthology.lock.ops:Node 'ubuntu@smithi090.front.sepia.ceph.com' reimaging is complete
2021-12-19T16:37:35.682 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging
2021-12-19T16:37:35.682 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server
2021-12-19T16:37:35.683 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2021-12-19T16:37:35.701 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete
2021-12-19T16:37:35.905 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of centos 8.3
2021-12-19T16:37:35.911 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of centos 8.3
2021-12-19T16:37:36.279 INFO:teuthology.orchestra.console:Power off smithi090
2021-12-19T16:37:36.365 INFO:teuthology.orchestra.console:Power off smithi116
2021-12-19T16:37:44.657 INFO:teuthology.orchestra.console:Power off for smithi090 completed
2021-12-19T16:37:44.729 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2021-12-19T16:37:44.759 INFO:teuthology.orchestra.console:Power on smithi090
2021-12-19T16:37:44.831 INFO:teuthology.orchestra.console:Power on smithi116
2021-12-19T16:37:53.203 INFO:teuthology.orchestra.console:Power on for smithi116 completed
2021-12-19T16:37:53.225 INFO:teuthology.orchestra.console:Power on for smithi090 completed
2021-12-19T16:37:53.305 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish
2021-12-19T16:37:53.327 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish
2021-12-19T16:42:43.022 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-19T16:42:43.309 INFO:teuthology.provision.fog.smithi116:Node is ready
2021-12-19T16:42:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com
2021-12-19T16:42:43.378 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116
2021-12-19T16:42:43.710 INFO:teuthology.provision.fog.smithi116:Deploy complete!
2021-12-19T16:43:10.441 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-19T16:43:10.787 INFO:teuthology.provision.fog.smithi090:Node is ready
2021-12-19T16:43:10.803 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090.front.sepia.ceph.com
2021-12-19T16:43:10.858 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90 smithi090.front.sepia.ceph.com smithi090
2021-12-19T16:43:11.184 INFO:teuthology.provision.fog.smithi090:Deploy complete!
2021-12-19T16:43:11.368 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com
2021-12-19T16:43:11.369 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-19T16:43:11.389 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com
2021-12-19T16:43:11.389 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-19T16:43:11.407 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server
2021-12-19T16:43:11.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2021-12-19T16:43:11.662 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64
2021-12-19T16:43:11.716 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Linux"
2021-12-19T16:43:11.716 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="8"
2021-12-19T16:43:11.717 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos"
2021-12-19T16:43:11.717 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora"
2021-12-19T16:43:11.717 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="8"
2021-12-19T16:43:11.717 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el8"
2021-12-19T16:43:11.718 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Linux 8"
2021-12-19T16:43:11.718 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2021-12-19T16:43:11.718 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-19T16:43:11.718 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/"
2021-12-19T16:43:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-12-19T16:43:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-12-19T16:43:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-12-19T16:43:11.720 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2021-12-19T16:43:11.741 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server
2021-12-19T16:43:11.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com'
2021-12-19T16:43:12.012 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64
2021-12-19T16:43:12.066 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Linux"
2021-12-19T16:43:12.066 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8"
2021-12-19T16:43:12.067 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos"
2021-12-19T16:43:12.067 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora"
2021-12-19T16:43:12.067 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8"
2021-12-19T16:43:12.068 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8"
2021-12-19T16:43:12.068 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Linux 8"
2021-12-19T16:43:12.068 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31"
2021-12-19T16:43:12.068 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-19T16:43:12.069 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/"
2021-12-19T16:43:12.069 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-12-19T16:43:12.069 INFO:teuthology.orchestra.run.smithi116.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-12-19T16:43:12.069 INFO:teuthology.orchestra.run.smithi116.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-12-19T16:43:12.070 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2021-12-19T16:43:12.179 INFO:teuthology.dispatcher.supervisor:Running job 6571147
2021-12-19T16:43:12.180 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-12-19_03:31:03-rados-pacific-distro-default-smithi/6571147 --name teuthology-2021-12-19_03:31:03-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2021-12-19_03:31:03-rados-pacific-distro-default-smithi/6571147/orig.config.yaml
2021-12-19T16:43:12.184 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-12-19_03:31:03-rados-pacific-distro-default-smithi/6571147
2021-12-19T16:43:12.185 INFO:teuthology.dispatcher.supervisor:Job PID: 18658
2021-12-19T16:43:12.185 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-20T04:44:05.242 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-20T04:44:05.441 INFO:teuthology.kill:Killing Pids: {18658}
2021-12-20T04:44:06.502 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-12-20T04:44:06.503 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-12-20T04:44:06.503 INFO:teuthology.misc:Compressing logs...
2021-12-20T04:44:06.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2021-12-20T04:44:06.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com'
2021-12-20T04:44:06.752 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-20T04:44:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-20T04:44:06.999 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-20T04:44:06.999 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now
2021-12-20T04:44:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-20T04:44:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:tar: Error is not recoverable: exiting now
2021-12-20T04:44:07.116 INFO:teuthology.misc:Compressing logs...
2021-12-20T04:44:07.367 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-20T04:44:07.535 INFO:teuthology.misc:Compressing logs...
2021-12-20T04:44:09.383 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-20T04:44:09.876 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-20T04:44:10.787 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-mgr.smithi090.ekxfen.log: file size changed while zipping
2021-12-20T04:44:10.933 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-20T04:44:12.714 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-20T04:44:13.008 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-mon.smithi090.log: file size changed while zipping
2021-12-20T04:44:13.266 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-mon.smithi116.log: file size changed while zipping
2021-12-20T04:44:14.739 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-20T04:44:14.841 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-20T04:44:16.255 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-20T04:44:18.579 INFO:teuthology.orchestra.run.smithi090.stderr:gzip: /var/log/ceph/172291cc-60ec-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-20T04:44:48.950 INFO:teuthology.orchestra.run.smithi090.stderr:tar: .: file changed as we read it
2021-12-20T04:45:14.447 INFO:teuthology.kill:No teuthology processes running
2021-12-20T04:45:15.520 INFO:teuthology.kill:Nuking machines: ['smithi090', 'smithi116']
2021-12-20T04:45:16.420 INFO:teuthology.kill:2021-12-20 04:45:16,420.420 INFO:teuthology.nuke:targets:
2021-12-20T04:45:16.421 INFO:teuthology.kill:  smithi090.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNMeOarY63NpIT9HiZIP6InGK8LrkMIr3RHbtDfa6oLSiih/7XRt08DFGLDuTM2OV4NrMY+wgIfA7fROXl0oGq8pt3DTKuxdlad8W3uxjBbwUMFp+GNOzWn7sy+bxBmlrb29te8sx80TQwVSfD924Z4gvryxr8/0/IoeAVZnJutiBENbki2If8dpI+dZcfnZlmoX5iNe0XWH7Lu71iGq1jKvDBNg6BPgSU+Gv5Xt9tO2ITrJhIJ8hJjt+XcCyITyrg21if98q13jURK0VA0j9azD/RKwKCFc9Mc8ayTiXckg4iYXo711AZCMFobHS9e3Gn+K21Eg84sU/lR+wKhmQFFQ9OwOtvJBfKe52Ywf+LOsO9lDKrY/K6Vz/H+yyxLc5OdI5Kk58qztFHdKr+DkV48L6r52GNnYtFvxu2IQ1a0eFxFeNYq9pL04aCiNhlUfOP/RSFAEUTsgK2cHc+dVDFwPoWgWYn/2V+Ngy6js0B8fTfjGT58IgJ38EQ5MQ3FM8=
2021-12-20T04:45:16.422 INFO:teuthology.kill:  smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDBBZZnu/gfma84BklVX64y9MHYfITGyJAr17n/UYahieQ4ZZKCfWVk09r3ZIO44JD9svF6gAIG0KcsvyC+z3lJ/YibvtC1fFD4uZ/Yj/TFpiDAitLL08X9ZDLMWqIVjqdlmC6lVTMKBuMqClw45gi9n3qRXRQAvJ9+p9wc1msFRICD8pco8Sd99Bgd9kk5OmliFIuvC7+jupo8xxCIVTrI32B/nTldufwdlTftjhQu0IuBLkwu+X5NbNfq9e/IhMdZ9FRsUls3NtOaNjoWY3riAn/NexRHv5pCLeAyUtPpTaHaSVJsbK8yVax9/Eh5GMc9DGRqelWmIwPnGMB+VvJE7zyICWKnOSAHC1xJiWSiuI5WzyXkKKiGKBS5fs+Z4pjhDvONqs0Jd46MitRiCzGrE/SRTQaDw/lKz8USiHFjJv42py8UjfJmMUHZ0BDaiYtkWIYQ+DGi3XPNd2jEUpHyB9IzuITsRJdVzPR3qhE4dbO05wtJ61ncdtAEAwZdls=
2021-12-20T04:45:16.446 INFO:teuthology.kill:2021-12-20 04:45:16,446.446 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-20T04:45:16.449 INFO:teuthology.kill:2021-12-20 04:45:16,449.449 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-20T04:45:16.523 INFO:teuthology.kill:2021-12-20 04:45:16,522.522 INFO:teuthology.orchestra.console:Power off smithi116
2021-12-20T04:45:16.535 INFO:teuthology.kill:2021-12-20 04:45:16,535.535 INFO:teuthology.orchestra.console:Power off smithi090
2021-12-20T04:45:24.894 INFO:teuthology.kill:2021-12-20 04:45:24,893.893 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2021-12-20T04:45:25.056 INFO:teuthology.kill:2021-12-20 04:45:25,055.055 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com
2021-12-20T04:45:29.012 INFO:teuthology.kill:2021-12-20 04:45:29,011.011 INFO:teuthology.orchestra.console:Power off for smithi090 completed
2021-12-20T04:45:29.157 INFO:teuthology.kill:2021-12-20 04:45:29,157.157 INFO:teuthology.lock.ops:unlocked: smithi090.front.sepia.ceph.com
2021-12-20T04:47:29.400 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15