2020-11-16T20:35:50.929 INFO:root:teuthology version: 1.0.0-357fd4a
2020-11-16T20:35:50.931 DEBUG:teuthology.run:Teuthology command: teuthology --name kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi --verbose --lock --archive /home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631163 --block --description rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} --owner scheduled_kchai@teuthology -- /tmp/teuthology-worker.9ytm1oxi.tmp
2020-11-16T20:35:50.956 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/
2020-11-16T20:35:51.024 INFO:teuthology.run:Config:
  archive_path: /home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631163
  branch: wip-kefu-testing-2020-11-16-1845
  description: rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity
    start}
  email: tchaikov@gmail.com
  first_in_suite: false
  job_id: '5631163'
  kernel:
    kdb: true
    sha1: distro
  last_in_suite: false
  machine_type: smithi
  name: kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi
  nuke-on-error: true
  openstack:
  - volumes:
      count: 4
      size: 10
  os_type: ubuntu
  os_version: '18.04'
  overrides:
    admin_socket:
      branch: wip-kefu-testing-2020-11-16-1845
    ceph:
      conf:
        global:
          mon election default strategy: 3
        mgr:
          debug mgr: 20
          debug ms: 1
        mon:
          debug mon: 20
          debug ms: 1
          debug paxos: 20
        osd:
          debug filestore: 20
          debug journal: 20
          debug ms: 20
          debug osd: 25
          osd shutdown pgref assert: true
      log-ignorelist:
      - \(MDS_ALL_DOWN\)
      - \(MDS_UP_LESS_THAN_MAX\)
      - Health detail
      log-whitelist:
      - \(MDS_ALL_DOWN\)
      - \(MDS_UP_LESS_THAN_MAX\)
      - Health detail
      sha1: 911cc35794d871071a8bebfb0919482263b1d18b
    ceph-deploy:
      conf:
        client:
          log file: /var/log/ceph/ceph-$name.$pid.log
        mon:
          osd default pool size: 2
    install:
      ceph:
        sha1: 911cc35794d871071a8bebfb0919482263b1d18b
    workunit:
      branch: wip-kefu-testing-2020-11-16-1845
      sha1: 911cc35794d871071a8bebfb0919482263b1d18b
  owner: scheduled_kchai@teuthology
  priority: 90
  repo: git://git.ceph.com/ceph-ci.git
  roles:
  - - mon.a
    - mon.c
    - mgr.y
    - osd.0
    - osd.1
    - osd.2
    - osd.3
    - client.0
    - ceph.rgw.realm.zone.a
    - node-exporter.a
    - alertmanager.a
  - - mon.b
    - mgr.x
    - osd.4
    - osd.5
    - osd.6
    - osd.7
    - client.1
    - prometheus.a
    - grafana.a
    - node-exporter.b
    - ceph.iscsi.iscsi.a
  sha1: 911cc35794d871071a8bebfb0919482263b1d18b
  sleep_before_teardown: 0
  suite: rados
  suite_branch: wip-kefu-testing-2020-11-16-1845
  suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-11-16-1845/qa
  suite_relpath: qa
  suite_repo: git://git.ceph.com/ceph-ci.git
  suite_sha1: 911cc35794d871071a8bebfb0919482263b1d18b
  tasks:
  - cephadm:
      conf:
        mgr:
          debug mgr: 20
          debug ms: 1
  - cephadm.shell:
      mon.a:
      - ceph orch status
      - ceph orch ps
      - ceph orch ls
      - ceph orch host ls
      - ceph orch device ls
  teuthology_branch: master
  timestamp: 2020-11-16_12:11:03
  tube: smithi
  user: kchai
  verbose: false
  worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.11573
2020-11-16T20:35:51.024 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-11-16-1845/qa; will attempt to use it
2020-11-16T20:35:51.025 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-11-16-1845/qa/tasks
2020-11-16T20:35:51.025 INFO:teuthology.run_tasks:Running task internal.check_packages...
2020-11-16T20:35:51.028 INFO:teuthology.task.internal:Checking packages...
2020-11-16T20:35:51.048 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '911cc35794d871071a8bebfb0919482263b1d18b'
2020-11-16T20:35:51.048 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch
2020-11-16T20:35:51.048 INFO:teuthology.packaging:ref: None
2020-11-16T20:35:51.049 INFO:teuthology.packaging:tag: None
2020-11-16T20:35:51.049 INFO:teuthology.packaging:branch: wip-kefu-testing-2020-11-16-1845
2020-11-16T20:35:51.049 INFO:teuthology.packaging:sha1: 911cc35794d871071a8bebfb0919482263b1d18b
2020-11-16T20:35:51.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-kefu-testing-2020-11-16-1845
2020-11-16T20:35:51.221 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-7219-g911cc357-1bionic
2020-11-16T20:35:51.221 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep...
2020-11-16T20:35:51.231 INFO:teuthology.task.internal:no buildpackages task found
2020-11-16T20:35:51.231 INFO:teuthology.run_tasks:Running task internal.lock_machines...
2020-11-16T20:35:51.385 INFO:teuthology.task.internal.lock_machines:Locking machines...
2020-11-16T20:35:51.386 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/
2020-11-16T20:35:51.441 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:36:01.454 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:36:11.464 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:36:21.473 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:36:31.491 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:36:41.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:36:51.517 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)...
2020-11-16T20:37:01.535 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)...
2020-11-16T20:37:11.547 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)...
2020-11-16T20:37:21.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)...
2020-11-16T20:37:31.578 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)...
2020-11-16T20:37:41.593 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:37:51.610 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:38:01.625 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:38:11.638 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:38:21.651 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:38:31.721 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:38:41.737 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:38:51.747 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:39:01.764 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:39:11.774 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:39:21.790 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:39:31.806 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:39:41.821 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:39:51.839 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:40:01.856 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:40:11.873 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:40:21.890 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:40:31.906 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:40:41.923 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:40:51.939 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:41:01.955 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:41:11.974 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:41:21.990 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:41:32.007 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:41:42.023 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:41:52.035 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:42:02.054 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:42:12.071 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:42:22.083 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:42:32.100 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:42:42.119 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:42:52.133 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:43:02.151 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:43:12.169 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:43:22.187 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:43:32.200 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:43:42.211 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:43:52.223 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:44:02.240 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:44:12.253 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:44:22.274 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:44:32.291 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:44:42.304 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:44:52.322 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:45:02.341 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:45:12.359 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:45:22.375 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)...
2020-11-16T20:45:32.391 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_kchai@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631163'}
2020-11-16T20:45:32.413 DEBUG:teuthology.lock.ops:locked ubuntu@smithi064.front.sepia.ceph.com, ubuntu@smithi197.front.sepia.ceph.com
2020-11-16T20:45:32.472 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fb509c51e10>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fb5085faea0>, signals=[15])
2020-11-16T20:45:32.686 INFO:teuthology.provision.fog.smithi197:Scheduling deploy of ubuntu 18.04
2020-11-16T20:45:32.689 INFO:teuthology.provision.fog.smithi064:Scheduling deploy of ubuntu 18.04
2020-11-16T20:45:32.974 INFO:teuthology.orchestra.console:Power off smithi197
2020-11-16T20:45:32.975 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2020-11-16T20:45:32.990 INFO:teuthology.orchestra.console:Power off smithi064
2020-11-16T20:45:32.991 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2020-11-16T20:45:33.000 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:33.011 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:37.027 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:37.037 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:41.138 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:45.178 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:45.381 INFO:teuthology.orchestra.console:Power off for smithi197 completed
2020-11-16T20:45:45.389 INFO:teuthology.orchestra.console:Power off for smithi064 completed
2020-11-16T20:45:45.483 INFO:teuthology.orchestra.console:Power on smithi197
2020-11-16T20:45:45.483 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2020-11-16T20:45:45.493 INFO:teuthology.orchestra.console:Power on smithi064
2020-11-16T20:45:45.494 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2020-11-16T20:45:45.504 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:45.515 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:49.528 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:49.538 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:49.740 INFO:teuthology.orchestra.console:Power on for smithi197 completed
2020-11-16T20:45:49.841 INFO:teuthology.provision.fog.smithi197:Waiting for deploy to finish
2020-11-16T20:45:53.650 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2020-11-16T20:45:57.895 INFO:teuthology.orchestra.console:Power on for smithi064 completed
2020-11-16T20:45:57.996 INFO:teuthology.provision.fog.smithi064:Waiting for deploy to finish
2020-11-16T20:48:43.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:48:50.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:08.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:15.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:17.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:24.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:26.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:33.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:35.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:42.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:49:44.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:50:48.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:50:48.506 INFO:teuthology.orchestra.run:Running command with timeout 600
2020-11-16T20:50:48.508 INFO:teuthology.orchestra.run.smithi197:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2020-11-16T20:50:48.867 INFO:teuthology.provision.fog.smithi197:Node is ready
2020-11-16T20:50:48.867 INFO:teuthology.orchestra.run.smithi197:> hostname
2020-11-16T20:50:48.913 INFO:teuthology.orchestra.run.smithi197.stdout:smithi197.front.sepia.ceph.com
2020-11-16T20:50:48.913 INFO:teuthology.orchestra.run.smithi197:> grep smithi197.front.sepia.ceph.com /etc/hosts
2020-11-16T20:50:48.961 INFO:teuthology.orchestra.run.smithi197.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197
2020-11-16T20:50:48.962 INFO:teuthology.orchestra.run.smithi197:> sudo hostname smithi197
2020-11-16T20:50:49.053 INFO:teuthology.orchestra.run.smithi197:> sudo sed -i -e 's/smithi197.front.sepia.ceph.com/smithi197/g' /etc/hosts
2020-11-16T20:50:49.068 INFO:teuthology.orchestra.run.smithi197:> sudo sed -i -e 's/172.21.15.197/172.21.15.197/g' /etc/hosts
2020-11-16T20:50:49.141 INFO:teuthology.orchestra.run.smithi197:> sudo sed -i -e 's/smithi197.front.sepia.ceph.com/smithi197/g' /etc/hostname
2020-11-16T20:50:49.199 INFO:teuthology.orchestra.run.smithi197:> sudo hostname smithi197
2020-11-16T20:50:49.252 INFO:teuthology.provision.fog.smithi197:Deploy complete!
2020-11-16T20:50:49.253 DEBUG:teuthology.parallel:result is None
2020-11-16T20:50:50.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:50:50.577 INFO:teuthology.orchestra.run:Running command with timeout 600
2020-11-16T20:50:50.577 INFO:teuthology.orchestra.run.smithi064:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2020-11-16T20:50:50.943 INFO:teuthology.provision.fog.smithi064:Node is ready
2020-11-16T20:50:50.943 INFO:teuthology.orchestra.run.smithi064:> hostname
2020-11-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064.front.sepia.ceph.com
2020-11-16T20:50:50.990 INFO:teuthology.orchestra.run.smithi064:> grep smithi064.front.sepia.ceph.com /etc/hosts
2020-11-16T20:50:51.038 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64 smithi064.front.sepia.ceph.com smithi064
2020-11-16T20:50:51.039 INFO:teuthology.orchestra.run.smithi064:> sudo hostname smithi064
2020-11-16T20:50:51.102 INFO:teuthology.orchestra.run.smithi064:> sudo sed -i -e 's/smithi064.front.sepia.ceph.com/smithi064/g' /etc/hosts
2020-11-16T20:50:51.160 INFO:teuthology.orchestra.run.smithi064:> sudo sed -i -e 's/172.21.15.64/172.21.15.64/g' /etc/hosts
2020-11-16T20:50:51.215 INFO:teuthology.orchestra.run.smithi064:> sudo sed -i -e 's/smithi064.front.sepia.ceph.com/smithi064/g' /etc/hostname
2020-11-16T20:50:51.270 INFO:teuthology.orchestra.run.smithi064:> sudo hostname smithi064
2020-11-16T20:50:51.326 INFO:teuthology.provision.fog.smithi064:Deploy complete!
2020-11-16T20:50:51.327 DEBUG:teuthology.parallel:result is None
2020-11-16T20:51:04.027 INFO:teuthology.lock.ops:Checking smithi064.front.sepia.ceph.com
2020-11-16T20:51:04.028 INFO:teuthology.lock.ops:New key found. Updating...
2020-11-16T20:51:04.046 INFO:teuthology.lock.ops:Checking smithi197.front.sepia.ceph.com
2020-11-16T20:51:04.046 INFO:teuthology.lock.ops:New key found. Updating...
2020-11-16T20:51:04.062 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: set os type and version on server
2020-11-16T20:51:04.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2020-11-16T20:51:04.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:51:04.138 INFO:teuthology.orchestra.run.smithi064:> true
2020-11-16T20:51:04.383 INFO:teuthology.orchestra.run.smithi064:> uname -m
2020-11-16T20:51:04.430 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64
2020-11-16T20:51:04.430 INFO:teuthology.orchestra.run.smithi064:> cat /etc/os-release
2020-11-16T20:51:04.478 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Ubuntu"
2020-11-16T20:51:04.478 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="18.04.5 LTS (Bionic Beaver)"
2020-11-16T20:51:04.478 INFO:teuthology.orchestra.run.smithi064.stdout:ID=ubuntu
2020-11-16T20:51:04.478 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE=debian
2020-11-16T20:51:04.478 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS"
2020-11-16T20:51:04.479 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="18.04"
2020-11-16T20:51:04.479 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.ubuntu.com/"
2020-11-16T20:51:04.479 INFO:teuthology.orchestra.run.smithi064.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2020-11-16T20:51:04.479 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2020-11-16T20:51:04.479 INFO:teuthology.orchestra.run.smithi064.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2020-11-16T20:51:04.479 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_CODENAME=bionic
2020-11-16T20:51:04.479 INFO:teuthology.orchestra.run.smithi064.stdout:UBUNTU_CODENAME=bionic
2020-11-16T20:51:04.480 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server
2020-11-16T20:51:04.496 INFO:teuthology.lock.ops:Updating [smithi197.front.sepia.ceph.com]: set os type and version on server
2020-11-16T20:51:04.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2020-11-16T20:51:04.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:51:04.572 INFO:teuthology.orchestra.run.smithi197:> true
2020-11-16T20:51:04.819 INFO:teuthology.orchestra.run.smithi197:> uname -m
2020-11-16T20:51:04.864 INFO:teuthology.orchestra.run.smithi197.stdout:x86_64
2020-11-16T20:51:04.865 INFO:teuthology.orchestra.run.smithi197:> cat /etc/os-release
2020-11-16T20:51:04.913 INFO:teuthology.orchestra.run.smithi197.stdout:NAME="Ubuntu"
2020-11-16T20:51:04.913 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION="18.04.5 LTS (Bionic Beaver)"
2020-11-16T20:51:04.913 INFO:teuthology.orchestra.run.smithi197.stdout:ID=ubuntu
2020-11-16T20:51:04.913 INFO:teuthology.orchestra.run.smithi197.stdout:ID_LIKE=debian
2020-11-16T20:51:04.913 INFO:teuthology.orchestra.run.smithi197.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS"
2020-11-16T20:51:04.914 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_ID="18.04"
2020-11-16T20:51:04.914 INFO:teuthology.orchestra.run.smithi197.stdout:HOME_URL="https://www.ubuntu.com/"
2020-11-16T20:51:04.914 INFO:teuthology.orchestra.run.smithi197.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2020-11-16T20:51:04.914 INFO:teuthology.orchestra.run.smithi197.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2020-11-16T20:51:04.914 INFO:teuthology.orchestra.run.smithi197.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2020-11-16T20:51:04.914 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_CODENAME=bionic
2020-11-16T20:51:04.914 INFO:teuthology.orchestra.run.smithi197.stdout:UBUNTU_CODENAME=bionic
2020-11-16T20:51:04.915 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server
2020-11-16T20:51:04.930 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far
2020-11-16T20:51:04.963 INFO:teuthology.task.internal.lock_machines:Locked targets:
  smithi064.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDwieZyS86bKWFanwMPtW2r232AYzxPiJ+dEzSo0/tWNxYBSxyg/r1HdK1YkXp3u4ppN2uH7BsDpjC3RprdzgU3lJ1dcD82FaQthVlOtMTjlAan9uAY46FlpzPcT9nPW0moRX3nKvz5kggz+vu0zr5OtIc0x7BQxBMmt5P25EGHCJyFUMwZ2/6oE1iyYbn04Wa781JYL8pJbkLoK25HHEnIRlBgSnMSGJGkkbxOpkzIYTohfcq2o8S0XYydIB+qR6myDVMtKqv1ohE+CVJb/Fl+8t/6dsawiBZSFvD2zGuWeMuz0h6Fj8mV+C+h5E1yoE/xG/RMzNMfDlRTJny+jJd1
  smithi197.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8SPx25NNpkLmAH1WWJiTPz3NNGOBUfBb5wHOvZQuEX65wCTvnwsAxWl5r76sTRUEyWDiUgo7xWNgrGXpsmvGxMbYfS0zQF1Tolx52pOPt4/TI977yX4oQAoyYIZTYE4i8odirgE4dwzsmuGx0Pn/Lf3e1UNO1z8wmZkJ/6LkslrSqLBhSEgojYUgNTAA7QLWV5gMy/kfgfmUV4rl2nloeGrWW+cOT8AbR/io4SFBTE3vB4NlxAR9dAez5sTNMMCQ2/ejURk1Ll9VqYkdCjl4q0HCz3p7lHfrbdmB4PjUEanccqPPA7hEu/8WPoiMM4rTcRtDQvLO/InjZ0BCBVxGF
2020-11-16T20:51:04.963 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/
2020-11-16T20:51:05.016 INFO:teuthology.run_tasks:Running task internal.save_config...
2020-11-16T20:51:05.025 INFO:teuthology.task.internal:Saving configuration
2020-11-16T20:51:05.036 INFO:teuthology.run_tasks:Running task internal.check_lock...
2020-11-16T20:51:05.051 INFO:teuthology.task.internal.check_lock:Checking locks...
2020-11-16T20:51:05.068 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 20:45:32.401452', 'description': '/home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631163', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN4rhLoPuxwvYCPxNP6N1uMwLz2AJQUS1i+k2Wx5l9FK', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi064.front.sepia.ceph.com'}
2020-11-16T20:51:05.084 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 20:45:32.401470', 'description': '/home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631163', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILCnlyyBerYBTS8g1Xe3pjPCPtwskQhwNkIPZMvxoFIB', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi197.front.sepia.ceph.com'}
2020-11-16T20:51:05.084 INFO:teuthology.run_tasks:Running task internal.add_remotes...
2020-11-16T20:51:05.092 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.realm.zone.a', 'node-exporter.a', 'alertmanager.a']
2020-11-16T20:51:05.092 INFO:teuthology.task.internal:roles: ubuntu@smithi197.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2020-11-16T20:51:05.092 INFO:teuthology.run_tasks:Running task console_log...
2020-11-16T20:51:05.157 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fb509c51e10>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fb5085bd950>, signals=[15])
2020-11-16T20:51:05.157 INFO:teuthology.run_tasks:Running task internal.connect...
2020-11-16T20:51:05.164 INFO:teuthology.task.internal:Opening connections...
2020-11-16T20:51:05.165 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com
2020-11-16T20:51:05.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:51:05.238 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi197.front.sepia.ceph.com
2020-11-16T20:51:05.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-11-16T20:51:05.315 INFO:teuthology.run_tasks:Running task internal.push_inventory...
2020-11-16T20:51:05.322 INFO:teuthology.orchestra.run.smithi064:> uname -m
2020-11-16T20:51:05.442 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64
2020-11-16T20:51:05.443 INFO:teuthology.orchestra.run.smithi064:> cat /etc/os-release
2020-11-16T20:51:05.489 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Ubuntu"
2020-11-16T20:51:05.490 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="18.04.5 LTS (Bionic Beaver)"
2020-11-16T20:51:05.490 INFO:teuthology.orchestra.run.smithi064.stdout:ID=ubuntu
2020-11-16T20:51:05.490 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE=debian
2020-11-16T20:51:05.490 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS"
2020-11-16T20:51:05.490 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="18.04"
2020-11-16T20:51:05.490 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.ubuntu.com/"
2020-11-16T20:51:05.491 INFO:teuthology.orchestra.run.smithi064.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2020-11-16T20:51:05.491 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2020-11-16T20:51:05.491 INFO:teuthology.orchestra.run.smithi064.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2020-11-16T20:51:05.491 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_CODENAME=bionic
2020-11-16T20:51:05.491 INFO:teuthology.orchestra.run.smithi064.stdout:UBUNTU_CODENAME=bionic
2020-11-16T20:51:05.492 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server
2020-11-16T20:51:05.507 INFO:teuthology.orchestra.run.smithi197:> uname -m
2020-11-16T20:51:05.519 INFO:teuthology.orchestra.run.smithi197.stdout:x86_64
2020-11-16T20:51:05.520 INFO:teuthology.orchestra.run.smithi197:> cat /etc/os-release
2020-11-16T20:51:05.567 INFO:teuthology.orchestra.run.smithi197.stdout:NAME="Ubuntu"
2020-11-16T20:51:05.567 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION="18.04.5 LTS (Bionic Beaver)"
2020-11-16T20:51:05.568 INFO:teuthology.orchestra.run.smithi197.stdout:ID=ubuntu
2020-11-16T20:51:05.568 INFO:teuthology.orchestra.run.smithi197.stdout:ID_LIKE=debian
2020-11-16T20:51:05.568 INFO:teuthology.orchestra.run.smithi197.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS"
2020-11-16T20:51:05.568 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_ID="18.04"
2020-11-16T20:51:05.568 INFO:teuthology.orchestra.run.smithi197.stdout:HOME_URL="https://www.ubuntu.com/"
2020-11-16T20:51:05.568 INFO:teuthology.orchestra.run.smithi197.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2020-11-16T20:51:05.569 INFO:teuthology.orchestra.run.smithi197.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2020-11-16T20:51:05.569 INFO:teuthology.orchestra.run.smithi197.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2020-11-16T20:51:05.569 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_CODENAME=bionic
2020-11-16T20:51:05.569 INFO:teuthology.orchestra.run.smithi197.stdout:UBUNTU_CODENAME=bionic
2020-11-16T20:51:05.569 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server
2020-11-16T20:51:05.586 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles...