2021-01-22T02:17:01.373 INFO:root:teuthology version: 1.0.0-fe79ba1b
2021-01-22T02:17:01.375 INFO:teuthology.results:subset: '971/9999'
2021-01-22T02:17:01.376 INFO:teuthology.results:seed: '2477'
2021-01-22T02:34:41.724 INFO:root:teuthology version: 1.0.0-fe79ba1b
2021-01-22T02:34:41.760 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-01-22T02:59:42.235 INFO:teuthology.results:Tests finished! gathering results...
2021-01-22T02:59:44.815 INFO:scrape:Found 34 jobs
2021-01-22T02:59:51.999 INFO:scrape:Found 6 distinct failure reasons
2021-01-22T02:59:52.000 INFO:scrape:Dead: b'2021-01-22T02:57:02.120 DEBUG:teuthology.exit:Finished running handlers'
2021-01-22T02:59:52.000 INFO:scrape:11 jobs: ['5808359', '5808363', '5808381', '5808383', '5808378', '5808351', '5808356', '5808361', '5808377', '5808353', '5808370']
2021-01-22T02:59:52.001 INFO:scrape:suites intersection: []
2021-01-22T02:59:52.001 INFO:scrape:suites union: ['1-install/nautilus', '1-install/nautilus-v2only', '1-install/octopus', '1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-partial-recovery}', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'backoff/normal', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'clusters/{openstack', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04_podman}', 'distro$/{ubuntu_18.04}', 'fast/normal', 'fixed-2', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v2only', 'objectstore/bluestore-stupid', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/cephadm/dashboard/{distro/centos_latest', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/cephadm/upgrade/{1-start', 'rados/singleton-nomsgr/{all/msgr', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}}', 'task/test_e2e}', 'thrashers/careful', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/none', 'thrashosds-health', 'three-plus-one}', 'workloads/ec-small-objects-many-deletes}', 'workloads/radosbench-high-concurrency}', 'workloads/rbd_cls}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}']
2021-01-22T02:59:52.002 INFO:scrape:
2021-01-22T02:59:52.002 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds
2021-01-22T02:59:52.002 INFO:scrape:5 jobs: ['5808350', '5808371', '5808362', '5808375', '5808368']
2021-01-22T02:59:52.003 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'fixed-2', 'rados/cephadm/upgrade/{1-start']
2021-01-22T02:59:52.003 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{centos_8.0}', 'distro$/{centos_latest}', 'distro$/{rhel_8.0}', 'distro$/{rhel_latest}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start']
2021-01-22T02:59:52.004 INFO:scrape:
2021-01-22T02:59:52.004 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi019 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'
2021-01-22T02:59:52.004 INFO:scrape:3 jobs: ['5808379', '5808352', '5808366']
2021-01-22T02:59:52.005 INFO:scrape:suites intersection: ['task/test_cephadm}']
2021-01-22T02:59:52.005 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}']
2021-01-22T02:59:52.006 INFO:scrape:
2021-01-22T02:59:52.006 INFO:scrape:Failure: Command failed on smithi194 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid 387a0a1c-5c5b-11eb-8f93-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.194 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring'
2021-01-22T02:59:52.007 INFO:scrape:10 jobs: ['5808365', '5808372', '5808355', '5808357', '5808382', '5808354', '5808360', '5808369', '5808373', '5808380']
2021-01-22T02:59:52.007 INFO:scrape:suites intersection: []
2021-01-22T02:59:52.008 INFO:scrape:suites union: ['fixed-2', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async-v1only', 'msgr/async-v2only', 'rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman', 'rados/cephadm/smoke/{distro/ubuntu_18.04_podman', 'rados/cephadm/with-work/{distro/ubuntu_18.04_podman', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'start', 'start}', 'task/test_orch_cli}', 'tasks/rados_python}']
2021-01-22T02:59:52.008 INFO:scrape:
2021-01-22T02:59:52.008 INFO:scrape:Failure: psutil.NoSuchProcess process no longer exists (pid=28807)
2021-01-22T02:59:52.009 INFO:scrape:4 jobs: ['5808358', '5808376', '5808374', '5808364']
2021-01-22T02:59:52.009 INFO:scrape:suites intersection: ['rados']
2021-01-22T02:59:52.010 INFO:scrape:suites union: ['1-install/nautilus', '1-install/nautilus-v2only', '1-install/octopus', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/singleton-nomsgr/{all/msgr', 'supported-random-distro$/{centos_8}}', 'thrashers/none', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}', 'workloads/radosbench}', 'workloads/rbd_cls}']
2021-01-22T02:59:52.010 INFO:scrape:
2021-01-22T02:59:52.011 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi115 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'
2021-01-22T02:59:52.011 INFO:scrape:1 jobs: ['5808367']
2021-01-22T02:59:52.011 INFO:scrape:suites: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}']
2021-01-22T02:59:52.012 INFO:scrape:
2021-01-22T02:59:52.013 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/
failed:  20
dead:    14
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[5808350]  rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/classic}
-----------------------------------------------------------------
time:   00:17:27
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808350/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808350/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d5e31fcde7224f9c994db85a11136ba6

    reached maximum tries (180) after waiting for 180 seconds

[5808352]  rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm}
-----------------------------------------------------------------
time:   00:07:51
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808352/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808352/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2f5d195a4e3f497293eb7ad8b611ee66

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi061 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=d328dbdcd10a95dc57dad0003bc0ec21e01f64e0
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[5808354]  rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:06:58
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808354/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808354/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7b6cbc04015c4835b4e89d77f5f2b3ce

    Command failed on smithi171 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    d436b686-5c5a-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.171 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'

[5808355]  rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:04:56
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808355/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808355/
sentry: https://sentry.ceph.com/organizations/ceph/?query=60dadd9f830642e8b8963905be0a20d4

    Command failed on smithi094 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    9f05e07c-5c5a-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.94 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'

[5808357]  rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_orch_cli}
-----------------------------------------------------------------
time:   00:05:19
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808357/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808357/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c069585518ef4b638d8db7b10fe7158d

    Command failed on smithi047 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    678f6a28-5c5a-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id a --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.47 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'

[5808358]  rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:25:11
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808358/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808358/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dcc72ef4ee4e422da71b219ad493ebfd

    psutil.NoSuchProcess process no longer exists (pid=28807)

[5808360]  rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start}
-----------------------------------------------------------------
time:   00:04:14
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808360/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808360/
sentry: https://sentry.ceph.com/organizations/ceph/?query=01a9f49c2242490388c7b180c9616462

    Command failed on smithi070 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    b4f0ad2c-5c5a-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.70 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'

[5808362]  rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/connectivity}
-----------------------------------------------------------------
time:   00:16:39
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808362/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808362/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f8ad5872fc0d42f6a888fb34fd35fe13

    reached maximum tries (180) after waiting for 180 seconds

[5808365]  rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:04:53
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808365/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808365/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dfab4ade50ae4009b4df6d172d78d0ee

    Command failed on smithi194 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    387a0a1c-5c5b-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.194 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'

[5808366]  rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm}
-----------------------------------------------------------------
time:   00:07:46
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808366/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808366/
sentry: https://sentry.ceph.com/organizations/ceph/?query=99e6c50e17d048c2abcd2eb6871865f9

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi023 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=d328dbdcd10a95dc57dad0003bc0ec21e01f64e0
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[5808367]  rados/cephadm/dashboard/{distro/centos_latest task/test_e2e}
-----------------------------------------------------------------
time:   00:15:56
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808367/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808367/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3e888512192e4c609ae84f54b7cf7578

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi115
    with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=d328dbdcd10a95dc57dad0003bc0ec21e01f64e0
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

[5808368]  rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/classic}
-----------------------------------------------------------------
time:   00:16:46
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808368/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808368/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9a7559af27d949e494b8e66fafd77097

    reached maximum tries (180) after waiting for 180 seconds

[5808369]  rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:06:59
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808369/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808369/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a685868e230d48f085273c3bea1440e1

    Command failed on smithi016 with status 1: "sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    8c402c6c-5c5b-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-
    daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.16:6789]' &&
    sudo chmod +r /etc/ceph/ceph.client.admin.keyring"

[5808371]  rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/connectivity}
-----------------------------------------------------------------
time:   00:17:31
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808371/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808371/
sentry: https://sentry.ceph.com/organizations/ceph/?query=721c6d491e8448d88316445a5ede7eaa

    reached maximum tries (180) after waiting for 180 seconds

[5808372]  rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_orch_cli}
-----------------------------------------------------------------
time:   00:05:21
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808372/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808372/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b4198921b3364c5d86eaa9b8870af2b0

    Command failed on smithi055 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    36f28642-5c5b-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id a --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.55 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'

[5808373]  rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:04:59
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808373/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808373/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6cf9733aaf974cd08c3e49676bd80ed3

    Command failed on smithi066 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    6c99e182-5c5b-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.66 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'

[5808375]  rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/classic}
-----------------------------------------------------------------
time:   00:18:37
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808375/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808375/
sentry: https://sentry.ceph.com/organizations/ceph/?query=67292f64d3d14a45b388f7c6d83a08aa

    reached maximum tries (180) after waiting for 180 seconds

[5808379]  rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm}
-----------------------------------------------------------------
time:   00:14:21
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808379/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808379/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7fa248ba197a4466863bdd01027be932

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi019 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=d328dbdcd10a95dc57dad0003bc0ec21e01f64e0
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[5808380]  rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start}
-----------------------------------------------------------------
time:   00:04:17
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808380/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808380/
sentry: https://sentry.ceph.com/organizations/ceph/?query=464b457590d349df9d6fd0acffcef5c8

    Command failed on smithi198 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    f86a183a-5c5b-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.198 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'

[5808382]  rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:04:54
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808382/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808382/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a69f375014be429b9682b23857741451

    Command failed on smithi062 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:d328dbdcd10a95dc57dad0003bc0ec21e01f64e0 -v bootstrap --fsid
    1fa27f32-5c5c-11eb-8f93-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.62 && sudo chmod +r
    /etc/ceph/ceph.client.admin.keyring'



Dead
=================================================================
[5808351]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/rbd_cls}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808351/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808351/

[5808353]  rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04_podman} fixed-2 mon_election/connectivity}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808353/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808353/

[5808356]  rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/classic}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808356/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808356/

[5808359]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808359/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808359/

[5808361]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/test_rbd_api}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808361/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808361/

[5808363]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808363/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808363/

[5808364]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:22:19
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808364/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808364/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7b3ea22d2b2543fa9784ce8b86866046

    psutil.NoSuchProcess process no longer exists (pid=4371)

[5808370]  rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808370/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808370/

[5808374]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/radosbench}
-----------------------------------------------------------------
time:   00:17:27
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808374/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808374/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bc14eb006f4c488b8a28913ac971f5de

    psutil.NoSuchProcess process no longer exists (pid=21142)

[5808376]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/rbd_cls}
-----------------------------------------------------------------
time:   00:15:51
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808376/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808376/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c621ce563638430f8e5b3a0f81495f2c

    psutil.NoSuchProcess process no longer exists (pid=23797)

[5808377]  rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808377/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808377/

[5808378]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808378/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808378/

[5808381]  rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/connectivity}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808381/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808381/

[5808383]  rados/cephadm/dashboard/{distro/centos_latest task/test_e2e}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808383/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2021-01-22_02:15:35-rados-wip-kefu-testing-2021-01-21-1119-distro-basic-smithi/5808383/