2021-12-19T18:35:37.854 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-19T18:35:37.978 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi016.front.sepia.ceph.com' reimaging 2021-12-19T18:35:37.979 INFO:teuthology.lock.ops:Updating [ubuntu@smithi016.front.sepia.ceph.com]: reset os type and version on server 2021-12-19T18:35:37.980 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2021-12-19T18:35:37.998 INFO:teuthology.lock.ops:Node 'ubuntu@smithi016.front.sepia.ceph.com' reimaging is complete 2021-12-19T18:35:37.999 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi157.front.sepia.ceph.com' reimaging 2021-12-19T18:35:37.999 INFO:teuthology.lock.ops:Updating [ubuntu@smithi157.front.sepia.ceph.com]: reset os type and version on server 2021-12-19T18:35:37.999 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2021-12-19T18:35:38.018 INFO:teuthology.lock.ops:Node 'ubuntu@smithi157.front.sepia.ceph.com' reimaging is complete 2021-12-19T18:35:38.247 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of centos 8.stream 2021-12-19T18:35:38.262 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of centos 8.stream 2021-12-19T18:35:38.751 INFO:teuthology.orchestra.console:Power off smithi016 2021-12-19T18:35:38.895 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2021-12-19T18:35:38.953 INFO:teuthology.orchestra.console:Power off smithi157 2021-12-19T18:35:38.997 INFO:teuthology.orchestra.console:Power on smithi016 2021-12-19T18:35:39.094 INFO:teuthology.orchestra.console:Power off for smithi157 completed 2021-12-19T18:35:39.195 INFO:teuthology.orchestra.console:Power on smithi157 2021-12-19T18:35:43.453 INFO:teuthology.orchestra.console:Power on for smithi157 completed 2021-12-19T18:35:43.554 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish 2021-12-19T18:35:47.483 INFO:teuthology.orchestra.console:Power on for smithi016 completed 2021-12-19T18:35:47.584 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2021-12-19T18:40:02.519 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-19T18:40:22.867 INFO:teuthology.provision.fog.smithi016:Node is ready 2021-12-19T18:40:22.895 INFO:teuthology.orchestra.run.smithi016.stdout:smithi126 2021-12-19T18:40:22.954 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.126 smithi126.front.sepia.ceph.com smithi126 2021-12-19T18:40:23.373 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2021-12-19T18:41:01.003 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-19T18:41:01.274 INFO:teuthology.provision.fog.smithi157:Node is ready 2021-12-19T18:41:01.331 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com 2021-12-19T18:41:01.389 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157 2021-12-19T18:41:01.732 INFO:teuthology.provision.fog.smithi157:Deploy complete! 2021-12-19T18:41:01.887 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2021-12-19T18:41:01.888 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-19T18:41:01.908 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com 2021-12-19T18:41:01.909 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-19T18:41:01.929 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2021-12-19T18:41:01.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2021-12-19T18:41:02.183 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2021-12-19T18:41:02.242 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2021-12-19T18:41:02.242 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8" 2021-12-19T18:41:02.243 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2021-12-19T18:41:02.243 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2021-12-19T18:41:02.243 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8" 2021-12-19T18:41:02.244 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2021-12-19T18:41:02.244 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-19T18:41:02.244 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2021-12-19T18:41:02.244 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-19T18:41:02.245 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2021-12-19T18:41:02.245 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-19T18:41:02.246 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-19T18:41:02.246 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-19T18:41:02.247 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2021-12-19T18:41:02.292 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server 2021-12-19T18:41:02.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2021-12-19T18:41:02.520 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2021-12-19T18:41:02.578 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2021-12-19T18:41:02.578 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8" 2021-12-19T18:41:02.579 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2021-12-19T18:41:02.579 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2021-12-19T18:41:02.579 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8" 2021-12-19T18:41:02.579 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2021-12-19T18:41:02.580 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-19T18:41:02.580 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2021-12-19T18:41:02.580 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-19T18:41:02.580 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2021-12-19T18:41:02.581 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-19T18:41:02.581 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-19T18:41:02.581 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-19T18:41:02.582 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2021-12-19T18:41:02.692 INFO:teuthology.dispatcher.supervisor:Running job 6572650 2021-12-19T18:41:02.693 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2021-12-19_18:26:29-rados-wip-bluestore-zero-detection-distro-default-smithi/6572650 --name lflores-2021-12-19_18:26:29-rados-wip-bluestore-zero-detection-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/lflores-2021-12-19_18:26:29-rados-wip-bluestore-zero-detection-distro-default-smithi/6572650/orig.config.yaml 2021-12-19T18:41:02.697 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2021-12-19_18:26:29-rados-wip-bluestore-zero-detection-distro-default-smithi/6572650 2021-12-19T18:41:02.698 INFO:teuthology.dispatcher.supervisor:Job PID: 32625 2021-12-19T18:41:02.699 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-20T06:41:55.096 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-20T06:41:55.243 INFO:teuthology.kill:Killing Pids: {32625} 2021-12-20T06:41:56.532 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-12-20T06:41:56.533 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-12-20T06:41:56.534 INFO:teuthology.misc:Compressing logs... 2021-12-20T06:41:56.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2021-12-20T06:41:56.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2021-12-20T06:41:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2021-12-20T06:41:56.968 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2021-12-20T06:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2021-12-20T06:41:57.011 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2021-12-20T06:41:57.057 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2021-12-20T06:41:57.058 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2021-12-20T06:41:57.059 INFO:teuthology.misc:Compressing logs... 2021-12-20T06:41:57.457 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-20T06:41:57.659 INFO:teuthology.misc:Compressing logs... 2021-12-20T06:42:03.405 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2021-12-20T06:42:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2021-12-20T06:42:08.341 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2021-12-20T06:42:09.928 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2021-12-20T06:42:13.903 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2021-12-20T06:42:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2021-12-20T06:42:18.473 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2021-12-20T06:42:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2021-12-20T06:42:21.278 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-mon.smithi157.log: file size changed while zipping 2021-12-20T06:42:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-mgr.smithi157.wuzfjg.log: file size changed while zipping 2021-12-20T06:42:27.050 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/090971fe-60fc-11ec-8c31-001a4aab830c/ceph-mon.smithi016.log: file size changed while zipping 2021-12-20T06:43:15.655 INFO:teuthology.kill:No teuthology processes running 2021-12-20T06:43:16.817 INFO:teuthology.kill:Nuking machines: ['smithi016', 'smithi157'] 2021-12-20T06:43:17.794 INFO:teuthology.kill:2021-12-20 06:43:17,794.794 INFO:teuthology.nuke:targets: 2021-12-20T06:43:17.795 INFO:teuthology.kill: smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYEvdNpl329abyd1iTVHh6//pAKJa5mFwQ71QbRWV/nrhdt4jHKnt6CI990NHnOywfzI4qt0k0EdOUTD+QvINAC4bxIX1qEu1AeOf3RVlLqrfUr+IoganFnzdm45NWva36e8r96BaiNcuv1Rew1sQABe/YkKdHPXHgYhMd59HsTcTp6v/qjDhajlzTdRr4Y0tZy8nu4ukiB7DXe0PykL6AUeP/ZRI6VutrosfsT+Ly2euvycmhkfaY1BsETe0zZEhhrcCXwf+feUpS43xr5eHc6j6NFJlCSZc0K/MzABMljDYJooT/ermnZ/QB0OM5fhLOf/oFjwLvlhueuX4QG9MFVkG/HCLIMmZp067NeGGXTkEqRFlK/xxUNxacC0OKxrDFJRSbqrqNn9sYyMAE742kdlvOmMD/EnJVQcjnTqyAiXeQI8e1psh00B+UMWZirQ4kYa0raYLR3MIgqdkAp2i9kIrCBgBfrOE1opx8e7KUYiCg6sKxuI+6uNPGlCYWgBk= 2021-12-20T06:43:17.796 INFO:teuthology.kill: smithi157.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ1Ob3UF6mUX8uYS2QOUOXClSunvnsSDQAWeGdqX/W1uNHygbsq2FM5P80qsvSnGSuj76HzoV33+XRczGer93Vi73gkLb7AiikBoHiL7/+fv5Dtp+PiGBtuJnBqv1TrZUqIt0jl8aXrHlLrVSQ/Q5vLbI9D0kOL4m2PLhAmfeFY9zGEmmh+AvXbAcI358ACMtAApXL1WLZJkkqwEMNkBat3IgJ9oeEPfMBJEDr3VkRst/tr+bBlJEoYr1fGjRrpapXP8e7UuwiHu0jjio5Hz5rP7jMkS6L+2Wl3I4sU/V0rh22WSCFFcuzEQfP71WdlXhQbugm/Z/+LlcGGxwebRqF63fyfTEryEo72lhUl4rYwjlMnHdXP/CZl9TTTGAPx4pkx47R8xjKEk173RPfO3Fx6JyX8SzSRA9GeObYwisUmuey2RT5xCCEiK3HRMc5NwXshF0P7r5SSHZSXUnx66YlaZcrIzaIVr6AvtD7CBqZPpaMi4ull1Te7Jda4yEEPTk= 2021-12-20T06:43:17.822 INFO:teuthology.kill:2021-12-20 06:43:17,822.822 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-20T06:43:17.826 INFO:teuthology.kill:2021-12-20 06:43:17,825.825 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-20T06:43:17.910 INFO:teuthology.kill:2021-12-20 06:43:17,909.909 INFO:teuthology.orchestra.console:Power off smithi016 2021-12-20T06:43:17.927 INFO:teuthology.kill:2021-12-20 06:43:17,926.926 INFO:teuthology.orchestra.console:Power off smithi157 2021-12-20T06:43:30.406 INFO:teuthology.kill:2021-12-20 06:43:30,405.405 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2021-12-20T06:43:30.412 INFO:teuthology.kill:2021-12-20 06:43:30,412.412 INFO:teuthology.orchestra.console:Power off for smithi157 completed 2021-12-20T06:43:30.550 INFO:teuthology.kill:2021-12-20 06:43:30,549.549 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com 2021-12-20T06:43:30.552 INFO:teuthology.kill:2021-12-20 06:43:30,552.552 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com 2021-12-20T06:45:30.824 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-20T06:45:30.945 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi016 but it was locked by another job: /home/teuthworker/archive/soumyakoduri-2021-12-20_06:33:16-rgw-wip-skoduri-dbstore-mp-distro-basic-smithi/6573147 2021-12-20T06:45:30.945 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi157 but it was locked by another job: /home/teuthworker/archive/soumyakoduri-2021-12-20_06:33:16-rgw-wip-skoduri-dbstore-mp-distro-basic-smithi/6573147