2021-11-14T13:32:53.606 INFO:root:teuthology version: 1.1.0-c56135d1
2021-11-14T13:32:53.728 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi050.front.sepia.ceph.com' reimaging
2021-11-14T13:32:53.729 INFO:teuthology.lock.ops:Updating [ubuntu@smithi050.front.sepia.ceph.com]: reset os type and version on server
2021-11-14T13:32:53.729 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2021-11-14T13:32:53.748 INFO:teuthology.lock.ops:Node 'ubuntu@smithi050.front.sepia.ceph.com' reimaging is complete
2021-11-14T13:32:53.749 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging
2021-11-14T13:32:53.750 INFO:teuthology.lock.ops:Updating [ubuntu@smithi092.front.sepia.ceph.com]: reset os type and version on server
2021-11-14T13:32:53.750 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server
2021-11-14T13:32:53.770 INFO:teuthology.lock.ops:Node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging is complete
2021-11-14T13:32:53.959 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 8.3
2021-11-14T13:32:53.978 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of centos 8.3
2021-11-14T13:32:54.317 INFO:teuthology.orchestra.console:Power off smithi050
2021-11-14T13:32:54.352 INFO:teuthology.orchestra.console:Power off smithi092
2021-11-14T13:32:54.462 INFO:teuthology.orchestra.console:Power off for smithi050 completed
2021-11-14T13:32:54.493 INFO:teuthology.orchestra.console:Power off for smithi092 completed
2021-11-14T13:32:54.564 INFO:teuthology.orchestra.console:Power on smithi050
2021-11-14T13:32:54.594 INFO:teuthology.orchestra.console:Power on smithi092
2021-11-14T13:32:58.820 INFO:teuthology.orchestra.console:Power on for smithi050 completed
2021-11-14T13:32:58.922 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish
2021-11-14T13:33:02.956 INFO:teuthology.orchestra.console:Power on for smithi092 completed
2021-11-14T13:33:03.057 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish
2021-11-14T13:37:33.679 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-14T13:37:34.027 INFO:teuthology.provision.fog.smithi050:Node is ready
2021-11-14T13:37:34.043 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com
2021-11-14T13:37:34.099 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050
2021-11-14T13:37:34.427 INFO:teuthology.provision.fog.smithi050:Deploy complete!
2021-11-14T13:37:53.009 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-14T13:37:53.303 INFO:teuthology.provision.fog.smithi092:Node is ready
2021-11-14T13:37:53.319 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com
2021-11-14T13:37:53.375 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092
2021-11-14T13:37:53.705 INFO:teuthology.provision.fog.smithi092:Deploy complete!
2021-11-14T13:37:53.863 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com
2021-11-14T13:37:53.864 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-14T13:37:53.885 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com
2021-11-14T13:37:53.885 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-14T13:37:53.906 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server
2021-11-14T13:37:53.907 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com'
2021-11-14T13:37:54.185 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64
2021-11-14T13:37:54.240 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Linux"
2021-11-14T13:37:54.240 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8"
2021-11-14T13:37:54.241 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos"
2021-11-14T13:37:54.241 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora"
2021-11-14T13:37:54.241 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8"
2021-11-14T13:37:54.241 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8"
2021-11-14T13:37:54.242 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Linux 8"
2021-11-14T13:37:54.242 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31"
2021-11-14T13:37:54.242 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-14T13:37:54.242 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/"
2021-11-14T13:37:54.242 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-11-14T13:37:54.243 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-11-14T13:37:54.243 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-11-14T13:37:54.244 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2021-11-14T13:37:54.271 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server
2021-11-14T13:37:54.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com'
2021-11-14T13:37:54.481 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64
2021-11-14T13:37:54.537 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Linux"
2021-11-14T13:37:54.537 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8"
2021-11-14T13:37:54.538 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos"
2021-11-14T13:37:54.538 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora"
2021-11-14T13:37:54.538 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8"
2021-11-14T13:37:54.538 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8"
2021-11-14T13:37:54.539 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Linux 8"
2021-11-14T13:37:54.539 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31"
2021-11-14T13:37:54.539 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-14T13:37:54.539 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/"
2021-11-14T13:37:54.540 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-11-14T13:37:54.540 INFO:teuthology.orchestra.run.smithi092.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-11-14T13:37:54.540 INFO:teuthology.orchestra.run.smithi092.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-11-14T13:37:54.541 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server
2021-11-14T13:37:54.676 INFO:teuthology.dispatcher.supervisor:Running job 6480537
2021-11-14T13:37:54.677 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_pdiazbou@teuthology --archive /home/teuthworker/archive/pdiazbou-2021-11-03_08:04:16-orch-wip-pereman2-testing-2021-11-01-2310-distro-basic-smithi/6480537 --name pdiazbou-2021-11-03_08:04:16-orch-wip-pereman2-testing-2021-11-01-2310-distro-basic-smithi --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/pdiazbou-2021-11-03_08:04:16-orch-wip-pereman2-testing-2021-11-01-2310-distro-basic-smithi/6480537/orig.config.yaml
2021-11-14T13:37:54.680 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdiazbou-2021-11-03_08:04:16-orch-wip-pereman2-testing-2021-11-01-2310-distro-basic-smithi/6480537
2021-11-14T13:37:54.680 INFO:teuthology.dispatcher.supervisor:Job PID: 30870
2021-11-14T13:37:54.681 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-15T01:38:49.539 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-15T01:38:49.749 INFO:teuthology.kill:Killing Pids: {30870}
2021-11-15T01:38:51.695 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-15T01:38:51.698 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-15T01:38:51.699 INFO:teuthology.misc:Compressing logs...
2021-11-15T01:38:51.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com'
2021-11-15T01:38:51.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com'
2021-11-15T01:38:52.209 INFO:teuthology.misc:Compressing logs...
2021-11-15T01:38:52.496 INFO:teuthology.misc:Compressing logs...
2021-11-15T01:39:01.912 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-mgr.smithi050.efupdw.log: file size changed while zipping
2021-11-15T01:39:08.240 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-11-15T01:39:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-mds.cephfs.smithi050.qydmxf.log: file size changed while zipping
2021-11-15T01:39:27.610 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-11-15T01:39:27.752 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-mgr.smithi092.utmedt.log: file size changed while zipping
2021-11-15T01:39:30.122 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-mon.smithi050.log: file size changed while zipping
2021-11-15T01:39:41.880 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-11-15T01:39:47.211 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-mds.cephfs.smithi092.szrvzb.log: file size changed while zipping
2021-11-15T01:39:47.229 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-11-15T01:40:02.545 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-11-15T01:40:20.688 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-11-15T01:40:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-mds.cephfs.smithi050.ndcnwz.log: file size changed while zipping
2021-11-15T01:40:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-mds.cephfs.smithi092.hrlltz.log: file size changed while zipping
2021-11-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/dec99f38-4551-11ec-8c2c-001a4aab830c/ceph-mon.smithi092.log: file size changed while zipping
2021-11-15T01:41:41.690 INFO:teuthology.kill:No teuthology processes running
2021-11-15T01:41:42.897 INFO:teuthology.kill:Nuking machines: ['smithi050', 'smithi092']
2021-11-15T01:41:43.902 INFO:teuthology.kill:2021-11-15 01:41:43,902.902 INFO:teuthology.nuke:targets:
2021-11-15T01:41:43.903 INFO:teuthology.kill:  smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzBwJ2bv/CRuwqI/2pwhdnFcPbu04B5RmLDlyxH164JiY4cJU3BSzswg1yhugqcAWj3Y4lDbPQvl0EVEQS29/zTGcjYeJV47F1R6c1lYJGu22iFKSv8q+YI6OwqwBr0jgU61l4KeHx+YsoVwXWZQJlbwZhCzXy5peP7cSMas6TJYJkqDXGn6VJ2h4s+L8PKOIejvuKE7O/R5BL5i+irG4InxYkgVLvmIWL6v6W2E7b0egGHWOVAa0Lz0jwtDKHexacGyu61aUjU/h1dXgvQNVuXQoqQpEddYKb19gW8AS2R2mMZTKmTb4EHiYEdkbzNCsNe0ZHtoaJrht6pItqNhp+kdWbO/ghxSzy/CXkx6Zy7nVo4vqWlOiAkUgnkLmttyNM9h95qLkY9UF6FSYXGmpQAfqth29+XWsSavH0ytMHX8G0Qnv/nQzQvrArUWM9F/NO5odnJ2uAvEjM6Y6lPLkhsfHTsXMirdBNNfT94jW6N+Nr7qnm+NekdQAl5FZjrk8=
2021-11-15T01:41:43.903 INFO:teuthology.kill:  smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnyVzwe/0jH7/oEJv0uGTT0VFa/q7HzcFepZK2h75FyTIw9U2l0RasyLtHt5ETj+wvLS7ueDf800rLjCyCv3fCi1VWqbRz0OHYG+B0+HM8uw/hUHwF5y2UPD5dvzm9diDGHeS8So5xaXiFL2M4FUD9qjBHXOqEZGx8vhacagCTapaq5ZgcJzWmGgv5ENtXqz+lWBVfczhpHjWrYrPOMJ5GVnfViqDSEfYkiAv8pI+B8P7OedbtOKjJ6wfLGyDjfJ5PglbQm36aofv2b/cJ4lKZYbABctzgK3dqHfZuoJpMqiEfCppW6MaQZi2hAvLzOaFwki6TIVQaKm6KeWPysU7zzBdyHZ9pmapKRVknNYY3NESU4OqDp8WzKLfqE+NLXZxk6wXvIiXTrbSg99Emc4DbDeyenzcKP8fCOzxNphLqqbOsp4d/TLycHaq/aL4OllTvVqGXwpsK9JXMqpXnOSg5oqAVkt8MzPtYluPd7x/lBWOjrPMphaOhAlRTKMtyHv8=
2021-11-15T01:41:43.931 INFO:teuthology.kill:2021-11-15 01:41:43,931.931 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-15T01:41:43.934 INFO:teuthology.kill:2021-11-15 01:41:43,934.934 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-15T01:41:44.011 INFO:teuthology.kill:2021-11-15 01:41:44,010.010 INFO:teuthology.orchestra.console:Power off smithi050
2021-11-15T01:41:44.024 INFO:teuthology.kill:2021-11-15 01:41:44,024.024 INFO:teuthology.orchestra.console:Power off smithi092
2021-11-15T01:41:52.381 INFO:teuthology.kill:2021-11-15 01:41:52,380.380 INFO:teuthology.orchestra.console:Power off for smithi050 completed
2021-11-15T01:41:52.526 INFO:teuthology.kill:2021-11-15 01:41:52,525.525 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com
2021-11-15T01:41:56.513 INFO:teuthology.kill:2021-11-15 01:41:56,513.513 INFO:teuthology.orchestra.console:Power off for smithi092 completed
2021-11-15T01:41:56.663 INFO:teuthology.kill:2021-11-15 01:41:56,663.663 INFO:teuthology.lock.ops:unlocked: smithi092.front.sepia.ceph.com
2021-11-15T01:43:56.969 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-15T01:43:57.084 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi050 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-11-14_03:31:02-rados-pacific-distro-default-smithi/6502180
2021-11-15T01:43:57.084 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi092 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-11-14_03:31:02-rados-pacific-distro-default-smithi/6502179