2021-10-28T20:09:28.227 INFO:root:teuthology version: 1.1.0-c56135d1
2021-10-28T20:09:28.228 INFO:teuthology.results:subset: '1/4'
2021-10-28T20:09:28.229 INFO:teuthology.results:seed: '754'
2021-10-28T20:52:26.239 INFO:root:teuthology version: 1.1.0-c56135d1
2021-10-28T20:52:26.245 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-10-29T08:53:22.903 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2021-10-29T08:53:31.065 INFO:scrape:Found 96 jobs
2021-10-29T08:53:44.985 INFO:scrape:Found 7 distinct failure reasons
2021-10-29T08:53:44.987 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds
2021-10-29T08:53:44.987 INFO:scrape:23 jobs: ['6465296', '6465363', '6465336', '6465334', '6465314', '6465353', '6465324', '6465292', '6465276', '6465345', '6465350', '6465320', '6465286', '6465357', '6465304', '6465287', '6465344', '6465283', '6465362', '6465308', '6465315', '6465301', '6465331']
2021-10-29T08:53:44.988 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start']
2021-10-29T08:53:44.988 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', '2-ops/rm-zap-wait}', '2-services/basic', '2-services/client-keyring', '2-services/iscsi', '2-services/mirror', '2-services/nfs', '2-services/nfs-ingress', '2-services/nfs-ingress-rgw', '2-services/nfs-ingress2', '2-services/nfs2', '2-services/rgw', '2-services/rgw-ingress', '3-final}', 'rados:cephadm/osds/{0-distro/centos_8.2_container_tools_3.0', 'rados:cephadm/osds/{0-distro/centos_8.3_container_tools_3.0', 'rados:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0', 'rados:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8', 'rados:cephadm/osds/{0-distro/ubuntu_20.04', 'rados:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0', 'rados:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0', 'rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0', 'rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8', 'rados:cephadm/smoke-roleless/{0-distro/ubuntu_20.04']
2021-10-29T08:53:44.989 INFO:scrape:
2021-10-29T08:53:44.989 INFO:scrape:Failure: Command failed on smithi191 with status 5: 'sudo systemctl stop ceph-914af832-3832-11ec-8c28-001a4aab830c@mon.b'
2021-10-29T08:53:44.990 INFO:scrape:46 jobs
2021-10-29T08:53:44.990 INFO:scrape:suites intersection: ['fixed-2']
2021-10-29T08:53:44.991 INFO:scrape:suites union: ['1-start', '2-thrash', '3-tasks/rados_api_tests', '3-tasks/radosbench', '3-tasks/small-objects', '3-tasks/snaps-few-objects', 'agent/off', 'agent/on', 'distro/centos_8.2_container_tools_3.0', 'distro/centos_8.3_container_tools_3.0', 'distro/rhel_8.4_container_tools_3.0', 'distro/rhel_8.4_container_tools_rhel8', 'distro/ubuntu_20.04', 'fixed-2', 'mode/packaged', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'rados:cephadm/smoke/{0-nvme-loop', 'rados:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0', 'rados:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0', 'rados:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0', 'rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0', 'rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8', 'rados:cephadm/with-work/{0-distro/ubuntu_20.04', 'root}', 'start', 'start}', 'tasks/rados_api_tests}', 'tasks/rados_python}']
2021-10-29T08:53:44.991 INFO:scrape:
2021-10-29T08:53:44.992 INFO:scrape:Dead: b'2021-10-29T08:52:00.161 DEBUG:teuthology.exit:Finished running handlers'
2021-10-29T08:53:44.992 INFO:scrape:10 jobs: ['6465340', '6465322', '6465294', '6465273', '6465309', '6465280', '6465302', '6465274', '6465328', '6465311']
2021-10-29T08:53:44.993 INFO:scrape:suites intersection: []
2021-10-29T08:53:44.993 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/16.2.5', '1-bootstrap/octopus', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '3-upgrade-with-workload', '3-verify}', '4-final}', '4-verify}}', '4-wait', '5-upgrade-ls', 'agent/off', 'agent/on', 'centos_8.3_container_tools_3.0', 'conf/{client', 'mds', 'mon', 'mon_election/classic}', 'mon_election/connectivity}', 'osd}', 'overrides/{pg-warn', 'rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados:cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0', 'rados:cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus', 'rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2021-10-29T08:53:44.994 INFO:scrape:
2021-10-29T08:53:44.994 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi016 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=434b1409634993c5a89bbdcd2c0af0f073acdf91 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'
2021-10-29T08:53:44.995 INFO:scrape:2 jobs: ['6465338', '6465290']
2021-10-29T08:53:44.995 INFO:scrape:suites intersection: ['rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0', 'task/test_cephadm}']
2021-10-29T08:53:44.996 INFO:scrape:suites union: ['agent/off', 'agent/on', 'mon_election/classic', 'mon_election/connectivity', 'rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0', 'task/test_cephadm}']
2021-10-29T08:53:44.996 INFO:scrape:
2021-10-29T08:53:44.997 INFO:scrape:Failure: Command failed on smithi173 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:434b1409634993c5a89bbdcd2c0af0f073acdf91 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6caf13aa-382d-11ec-8c28-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4'
2021-10-29T08:53:44.997 INFO:scrape:6 jobs: ['6465318', '6465307', '6465278', '6465356', '6465366', '6465326']
2021-10-29T08:53:44.998 INFO:scrape:suites intersection: []
2021-10-29T08:53:44.998 INFO:scrape:suites union: ['1-start', '2-services/basic', '2-services/rgw', '3-final}', 'agent/off', 'agent/on', 'mon_election/classic', 'mon_election/connectivity', 'rados:cephadm/smoke-singlehost/{0-distro$/{centos_8.3_container_tools_3.0}', 'rados:cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8}', 'rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0', 'task/test_nfs}', 'task/test_orch_cli}']
2021-10-29T08:53:44.999 INFO:scrape:
2021-10-29T08:53:44.999 INFO:scrape:Dead: b'2021-10-29T08:53:36.834 INFO:journalctl@ceph.osd.2.smithi039.stdout:Oct 29 08:53:36 smithi039 bash[20170]: debug 2021-10-29T08:53:36.344+0000 7fd069dfe700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2021-10-29T07:53:36.347769+0000)'
2021-10-29T08:53:45.000 INFO:scrape:2 jobs: ['6465359', '6465351']
2021-10-29T08:53:45.000 INFO:scrape:suites intersection: []
2021-10-29T08:53:45.001 INFO:scrape:suites union: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '2-repo_digest/repo_digest', '3-start-upgrade', '3-upgrade-with-workload', '3-verify}', '4-verify}}', '4-wait', '5-upgrade-ls', 'agent/off', 'centos_8.3_container_tools_3.0', 'conf/{client', 'mds', 'mon', 'mon_election/connectivity}', 'osd}', 'overrides/{pg-warn', 'rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2021-10-29T08:53:45.001 INFO:scrape:
2021-10-29T08:53:45.002 INFO:scrape:Dead: b'2021-10-29T08:53:39.139 INFO:journalctl@ceph.mon.b.smithi142.stdout:Oct 29 08:53:38 smithi142 bash[35620]: audit 2021-10-29T08:53:38.575908+0000 mon.a (mon.0) 37952 : audit [DBG] from=\'client.87127 172.21.15.31:0/2706296153\' entity=\'client.admin\' cmd=[{"prefix": "versions"}]: dispatch'
2021-10-29T08:53:45.003 INFO:scrape:1 jobs: ['6465342']
2021-10-29T08:53:45.003 INFO:scrape:suites: ['2-repo_digest/defaut', '3-start-upgrade', '4-wait', '5-upgrade-ls', 'agent/on', 'mon_election/classic}', 'rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9']
2021-10-29T08:53:45.004 INFO:scrape:
2021-10-29T08:53:45.007 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/
failed:  77
dead:    9
running: 4
waiting: 0
queued:  0
passed:  6


Fail
=================================================================
[6465272]  rados:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python}
-----------------------------------------------------------------
time:   00:30:55
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465272/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465272/
sentry: https://sentry.ceph.com/organizations/ceph/?query=79b8727404304da8977fb3987713a597

    Command failed on smithi154 with status 5: 'sudo systemctl stop
    ceph-f3aab92c-382d-11ec-8c28-001a4aab830c@mon.b'

[6465276]  rados:cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:25:20
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465276/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465276/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c340c336819d43ab870c19164f74bd87

    reached maximum tries (180) after waiting for 180 seconds

[6465277]  rados:cephadm/smoke/{0-nvme-loop agent/on distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:24:24
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465277/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465277/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cd6eb2b7a42847bbb7531657d16960f1

    Command failed on smithi186 with status 5: 'sudo systemctl stop
    ceph-6be7bf62-382d-11ec-8c28-001a4aab830c@mon.b'

[6465278]  rados:cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   00:13:47
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465278/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465278/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a13d7a63a5c1467cb70677ba2f359f17

    Command failed on smithi194 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:434b1409634993c5a89bbdcd2c0af0f073acdf91 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    1d16f8f8-382d-11ec-8c28-001a4aab830c -- ceph-volume lvm zap
    /dev/vg_nvme/lv_4'

[6465279]  rados:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:27:07
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465279/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465279/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b47cd6e7ce6e41119f1f4b33d9369ac4

    Command failed on smithi178 with status 5: 'sudo systemctl stop
    ceph-2148962e-382e-11ec-8c28-001a4aab830c@mon.b'

[6465281]  rados:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:26:45
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465281/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465281/
sentry: https://sentry.ceph.com/organizations/ceph/?query=19b0f17d019f4c3fa8b06e101ee041c2

    Command failed on smithi197 with status 5: 'sudo systemctl stop
    ceph-2051586e-382e-11ec-8c28-001a4aab830c@mon.b'

[6465283]  rados:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final}
-----------------------------------------------------------------
time:   00:29:15
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465283/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465283/
sentry: https://sentry.ceph.com/organizations/ceph/?query=732ef22316aa43c3bfd283498babbc5f

    reached maximum tries (180) after waiting for 180 seconds

[6465284]  rados:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:26:54
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465284/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465284/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0fb5b400060344f2b8bc167f6f7f73a8

    Command failed on smithi148 with status 5: 'sudo systemctl stop
    ceph-4c11e9fa-382e-11ec-8c28-001a4aab830c@mon.b'

[6465285]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:34:45
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465285/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465285/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ca961d64f81449b4a055eb1b8ea397de

    Command failed on smithi093 with status 5: 'sudo systemctl stop
    ceph-d8833e98-382e-11ec-8c28-001a4aab830c@mon.b'

[6465286]  rados:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw 3-final}
-----------------------------------------------------------------
time:   00:25:05
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465286/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465286/
sentry: https://sentry.ceph.com/organizations/ceph/?query=97d6a3c6c8d548799571e32322f18f6d

    reached maximum tries (180) after waiting for 180 seconds

[6465287]  rados:cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:27:02
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465287/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465287/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f80b92d7fcaa45bcbf7d37322afbd0fb

    reached maximum tries (180) after waiting for 180 seconds

[6465288]  rados:cephadm/smoke/{0-nvme-loop agent/off distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:25:29
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465288/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465288/
sentry: https://sentry.ceph.com/organizations/ceph/?query=da7614c063d143b3afff5d24d9c19c89

    Command failed on smithi140 with status 5: 'sudo systemctl stop ceph-
    cbbec098-382d-11ec-8c28-001a4aab830c@mon.b'

[6465289]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:34:53
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465289/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465289/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1d270f0230284169800557c591bc6d78

    Command failed on smithi099 with status 5: 'sudo systemctl stop
    ceph-172a4d12-382f-11ec-8c28-001a4aab830c@mon.b'

[6465290]  rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/off mon_election/connectivity task/test_cephadm}
-----------------------------------------------------------------
time:   00:18:34
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465290/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465290/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d6f0190336c945109bae97491dcb0687

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi102 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=434b1409634993c5a89bbdcd2c0af0f073acdf91
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[6465291]  rados:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:33:28
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465291/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465291/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2e528a385f2a4441ba59c4c50bf72645

    Command failed on smithi174 with status 5: 'sudo systemctl stop ceph-
    bb2d1f9a-382d-11ec-8c28-001a4aab830c@mon.b'

[6465292]  rados:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
-----------------------------------------------------------------
time:   00:25:30
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465292/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465292/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0b7da4ac928b492f8d3a693a49bc96c4

    reached maximum tries (180) after waiting for 180 seconds

[6465293]  rados:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:26:55
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465293/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465293/
sentry: https://sentry.ceph.com/organizations/ceph/?query=22009af2afc343a9b29f51b68ebd3a4a

    Command failed on smithi097 with status 5: 'sudo systemctl stop
    ceph-0cd5eb42-382e-11ec-8c28-001a4aab830c@mon.b'

[6465295]  rados:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python}
-----------------------------------------------------------------
time:   00:27:36
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465295/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465295/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7d85a20d6e47421396d98afa621c8896

    Command failed on smithi134 with status 5: 'sudo systemctl stop
    ceph-f7b3ebd8-382d-11ec-8c28-001a4aab830c@mon.b'

[6465296]  rados:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:28:35
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465296/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465296/
sentry: https://sentry.ceph.com/organizations/ceph/?query=aa273fc905a94fae98af4bb74ee1581b

    reached maximum tries (180) after waiting for 180 seconds

[6465297]  rados:cephadm/smoke/{0-nvme-loop agent/on distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:28:16
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465297/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465297/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4a75ec6c129f4f47992d168193d66f0e

    Command failed on smithi155 with status 5: 'sudo systemctl stop ceph-
    ae807544-382d-11ec-8c28-001a4aab830c@mon.b'

[6465298]  rados:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:26:49
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465298/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465298/
sentry: https://sentry.ceph.com/organizations/ceph/?query=56bc07091050488aa205c47a37ceed41

    Command failed on smithi172 with status 5: 'sudo systemctl stop
    ceph-47b750b6-382e-11ec-8c28-001a4aab830c@mon.b'

[6465300]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:34:24
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465300/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465300/
sentry: https://sentry.ceph.com/organizations/ceph/?query=af8797dadd2d4690a1c6e91633e618ea

    Command failed on smithi187 with status 5: 'sudo systemctl stop
    ceph-06becf48-382f-11ec-8c28-001a4aab830c@mon.b'

[6465301]  rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
-----------------------------------------------------------------
time:   00:28:25
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465301/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465301/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a8485527b7ee43968fd49dc14c578096

    reached maximum tries (180) after waiting for 180 seconds

[6465303]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:34:56
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465303/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465303/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f472f1cf140a4f1b9c493d22c031923a

    Command failed on smithi167 with status 5: 'sudo systemctl stop
    ceph-2996faea-382f-11ec-8c28-001a4aab830c@mon.b'

[6465304]  rados:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:29:31
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465304/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465304/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d7be333e20d54e1c837e710e6d481d5c

    reached maximum tries (180) after waiting for 180 seconds

[6465305]  rados:cephadm/smoke/{0-nvme-loop agent/off distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:27:41
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465305/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465305/
sentry: https://sentry.ceph.com/organizations/ceph/?query=34b2c5381b5b4776a7cd6706f6efe6fc

    Command failed on smithi123 with status 5: 'sudo systemctl stop
    ceph-662550d0-382d-11ec-8c28-001a4aab830c@mon.b'

[6465306]  rados:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:29:44
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465306/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465306/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8af4dd70aed54523a62542a375a3b9fe

    Command failed on smithi077 with status 5: 'sudo systemctl stop
    ceph-f6aef5b6-382d-11ec-8c28-001a4aab830c@mon.b'

[6465307]  rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/off mon_election/connectivity task/test_nfs}
-----------------------------------------------------------------
time:   00:15:36
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465307/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465307/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ad8e31ce8b1e4555b72576201ca5ae57

    Command failed on smithi016 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:434b1409634993c5a89bbdcd2c0af0f073acdf91 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    fb707b7e-382d-11ec-8c28-001a4aab830c -- ceph-volume lvm zap
    /dev/vg_nvme/lv_4'

[6465308]  rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final}
-----------------------------------------------------------------
time:   00:30:39
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465308/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465308/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c9be7af6d61a467a845c696c617b667c

    reached maximum tries (180) after waiting for 180 seconds

[6465310]  rados:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root}
-----------------------------------------------------------------
time:   00:26:26
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465310/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465310/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cd9b118f789e41f88c0425985da827ac

    Command failed on smithi052 with status 5: 'sudo systemctl stop
    ceph-6c7ce9ca-382d-11ec-8c28-001a4aab830c@mon.b'

[6465312]  rados:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:26:40
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465312/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465312/
sentry: https://sentry.ceph.com/organizations/ceph/?query=edd1308a1c6247978d210809f2261204

    Command failed on smithi143 with status 5: 'sudo systemctl stop
    ceph-432fbeb6-382e-11ec-8c28-001a4aab830c@mon.b'

[6465313]  rados:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python}
-----------------------------------------------------------------
time:   00:27:30
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465313/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465313/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6cd3779bfdc841fc95015e01d05a8990

    Command failed on smithi041 with status 5: 'sudo systemctl stop
    ceph-56b495d8-382e-11ec-8c28-001a4aab830c@mon.b'

[6465314]  rados:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final}
-----------------------------------------------------------------
time:   00:30:36
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465314/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465314/
sentry: https://sentry.ceph.com/organizations/ceph/?query=06d8a1ed9dba4221bf986d8894a6fdc6

    reached maximum tries (180) after waiting for 180 seconds

[6465315]  rados:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:30:10
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465315/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465315/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b8361b90c56b4757aa0828040771e7bc

    reached maximum tries (180) after waiting for 180 seconds

[6465316]  rados:cephadm/smoke/{0-nvme-loop agent/on distro/ubuntu_20.04 fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:30:09
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465316/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465316/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c08d20fd31134e21b867e3ccfbc9eb00

    Command failed on smithi125 with status 5: 'sudo systemctl stop
    ceph-9451b674-382d-11ec-8c28-001a4aab830c@mon.b'

[6465317]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:33:44
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465317/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465317/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b92d5504d8b49eab79473c40e7ba055

    Command failed on smithi183 with status 5: 'sudo systemctl stop
    ceph-093486be-382f-11ec-8c28-001a4aab830c@mon.b'

[6465318]  rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/on mon_election/classic task/test_orch_cli}
-----------------------------------------------------------------
time:   00:10:56
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465318/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465318/
sentry: https://sentry.ceph.com/organizations/ceph/?query=43c02ff91adc478084ae4e993a94ea2c

    Command failed on smithi173 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:434b1409634993c5a89bbdcd2c0af0f073acdf91 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    6caf13aa-382d-11ec-8c28-001a4aab830c -- ceph-volume lvm zap
    /dev/vg_nvme/lv_4'

[6465319]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:34:49
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465319/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465319/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2e2fbe757d114dc58c7c8b23486d80b6

    Command failed on smithi179 with status 5: 'sudo systemctl stop
    ceph-24bfbfca-382f-11ec-8c28-001a4aab830c@mon.b'

[6465320]  rados:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
-----------------------------------------------------------------
time:   00:26:32
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465320/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465320/
sentry: https://sentry.ceph.com/organizations/ceph/?query=11ac16d98cae4c3a829ce4f89e71bb52

    reached maximum tries (180) after waiting for 180 seconds

[6465321]  rados:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:32:35
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465321/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465321/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b8a1f8efa89f4f3c9f7fc9cc3d7148d8

    Command failed on smithi145 with status 5: 'sudo systemctl stop ceph-
    bb16aa30-382d-11ec-8c28-001a4aab830c@mon.b'

[6465324]  rados:cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:26:21
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465324/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465324/
sentry: https://sentry.ceph.com/organizations/ceph/?query=486faa7c578c4e7a9501352df722b4ac

    reached maximum tries (180) after waiting for 180 seconds

[6465325]  rados:cephadm/smoke/{0-nvme-loop agent/off distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   01:16:12
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465325/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465325/
sentry: https://sentry.ceph.com/organizations/ceph/?query=17d131662310444ab485336992bcd815

    Command failed on smithi059 with status 5: 'sudo systemctl stop
    ceph-e18e8dcc-382d-11ec-8c28-001a4aab830c@mon.b'

[6465326]  rados:cephadm/smoke-singlehost/{0-distro$/{centos_8.3_container_tools_3.0} 1-start 2-services/rgw 3-final}
-----------------------------------------------------------------
time:   00:13:09
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465326/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465326/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a661097ba7364e20bddcabea64b7a883

    Command failed on smithi017 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:434b1409634993c5a89bbdcd2c0af0f073acdf91 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    977b5ba2-382d-11ec-8c28-001a4aab830c -- ceph-volume lvm zap
    /dev/vg_nvme/lv_4'

[6465327]  rados:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:27:02
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465327/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465327/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f16c798f69b24a139c283a914699cead

    Command failed on smithi138 with status 5: 'sudo systemctl stop
    ceph-488779e4-382e-11ec-8c28-001a4aab830c@mon.b'

[6465329]  rados:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:27:45
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465329/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465329/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bb1742c7b07f4c48a67c094ec31b819b

    Command failed on smithi159 with status 5: 'sudo systemctl stop
    ceph-c3631d9a-382d-11ec-8c28-001a4aab830c@mon.b'

[6465331]  rados:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final}
-----------------------------------------------------------------
time:   00:26:22
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465331/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465331/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6167fe6daf75492a8d35a4498164ab53

    reached maximum tries (180) after waiting for 180 seconds

[6465332]  rados:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   01:17:09
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465332/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465332/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9aef35c8fef043199eaf7057e7b1330f

    Command failed on smithi150 with status 5: 'sudo systemctl stop
    ceph-54dc9f8a-382e-11ec-8c28-001a4aab830c@mon.b'

[6465333]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python}
-----------------------------------------------------------------
time:   00:34:02
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465333/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465333/
sentry: https://sentry.ceph.com/organizations/ceph/?query=19e32f348abf4b5a98dd8b9f5d7f5f6e

    Command failed on smithi132 with status 5: 'sudo systemctl stop
    ceph-2803c492-382f-11ec-8c28-001a4aab830c@mon.b'

[6465334]  rados:cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:26:08
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465334/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465334/
sentry: https://sentry.ceph.com/organizations/ceph/?query=68edecff018740cab551d6221a8d59b2

    reached maximum tries (180) after waiting for 180 seconds

[6465335]  rados:cephadm/smoke/{0-nvme-loop agent/on distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:19:05
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465335/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465335/
sentry: https://sentry.ceph.com/organizations/ceph/?query=606dd3b6b64040178418c4c81861a706

    Command failed on smithi173 with status 5: 'sudo systemctl stop
    ceph-16d33eae-3830-11ec-8c28-001a4aab830c@mon.b'

[6465336]  rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   01:12:28
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465336/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465336/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8868407afc0845da983e41018777c782

    reached maximum tries (180) after waiting for 180 seconds

[6465337]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:33:21
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465337/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465337/
sentry: https://sentry.ceph.com/organizations/ceph/?query=98a03e0918954966a710ec0b24289f1b

    Command failed on smithi191 with status 5: 'sudo systemctl stop
    ceph-914af832-3832-11ec-8c28-001a4aab830c@mon.b'

[6465338]  rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/on mon_election/classic task/test_cephadm}
-----------------------------------------------------------------
time:   00:14:22
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465338/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465338/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0afaafadedbb4253a1a6f8efb0ce9902

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi016 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=434b1409634993c5a89bbdcd2c0af0f073acdf91
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[6465339]  rados:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:24:57
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465339/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465339/
sentry: https://sentry.ceph.com/organizations/ceph/?query=83ca665f3e374d47912e7591b3177c33

    Command failed on smithi186 with status 5: 'sudo systemctl stop
    ceph-4b65eb38-3832-11ec-8c28-001a4aab830c@mon.b'

[6465341]  rados:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:22:45
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465341/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465341/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b64617b4dc9f42a1b782e9837788aac7

    Command failed on smithi052 with status 5: 'sudo systemctl stop
    ceph-99c190b6-3832-11ec-8c28-001a4aab830c@mon.b'

[6465343]  rados:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:21:30
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465343/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465343/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b49b6cc3cdf24799b91c2b591ea67413

    Command failed on smithi196 with status 5: 'sudo systemctl stop
    ceph-b7427d9e-3832-11ec-8c28-001a4aab830c@mon.b'

[6465344]  rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final}
-----------------------------------------------------------------
time:   00:23:58
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465344/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465344/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b7077dceeea34a89abcd63315b86925a

    reached maximum tries (180) after waiting for 180 seconds

[6465345]  rados:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:24:15
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465345/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465345/
sentry: https://sentry.ceph.com/organizations/ceph/?query=72f87e1e44774e288942158fb0d48042

    reached maximum tries (180) after waiting for 180 seconds

[6465346]  rados:cephadm/smoke/{0-nvme-loop agent/off distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:23:15
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465346/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465346/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ec124c3654ca4ec9b6c5131eb4daef0a

    Command failed on smithi155 with status 5: 'sudo systemctl stop
    ceph-9dd2d624-3832-11ec-8c28-001a4aab830c@mon.b'

[6465347]  rados:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:22:12
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465347/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465347/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f5859dcdbb914a63aaaa430cbe81e2d9

    Command failed on smithi190 with status 5: 'sudo systemctl stop
    ceph-1c8afa5a-3833-11ec-8c28-001a4aab830c@mon.b'

[6465349]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:33:09
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465349/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465349/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e989415ae8434a5aa34846e8fe741aee

    Command failed on smithi104 with status 5: 'sudo systemctl stop
    ceph-367e4d58-3834-11ec-8c28-001a4aab830c@mon.b'

[6465350]  rados:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final}
-----------------------------------------------------------------
time:   00:21:42
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465350/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465350/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fae36d04e56248c68b97c867ae67773c

    reached maximum tries (180) after waiting for 180 seconds

[6465352]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python}
-----------------------------------------------------------------
time:   00:33:55
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465352/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465352/
sentry: https://sentry.ceph.com/organizations/ceph/?query=66269cd959824aa09448a942259f984f

    Command failed on smithi192 with status 5: 'sudo systemctl stop
    ceph-32b6c3da-3834-11ec-8c28-001a4aab830c@mon.b'

[6465353]  rados:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:24:24
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465353/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465353/
sentry: https://sentry.ceph.com/organizations/ceph/?query=66ca1a356bac4f5f99d4d278e7de3542

    reached maximum tries (180) after waiting for 180 seconds

[6465354]  rados:cephadm/smoke/{0-nvme-loop agent/on distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:23:30
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465354/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465354/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4bc75945e8e94673bc98e856937a957d

    Command failed on smithi140 with status 5: 'sudo systemctl stop ceph-
    ce62d4f6-3832-11ec-8c28-001a4aab830c@mon.b'

[6465355]  rados:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:24:53
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465355/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465355/
sentry: https://sentry.ceph.com/organizations/ceph/?query=81db0bab92a84ecaa7e1472b63327e30

    Command failed on smithi084 with status 5: 'sudo systemctl stop
    ceph-d9a1f2c0-3832-11ec-8c28-001a4aab830c@mon.b'

[6465356]  rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/on mon_election/classic task/test_nfs}
-----------------------------------------------------------------
time:   00:10:47
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465356/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465356/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d5074972d25840608e1fa58b9a384e51

    Command failed on smithi148 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:434b1409634993c5a89bbdcd2c0af0f073acdf91 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    ed0787b2-3832-11ec-8c28-001a4aab830c -- ceph-volume lvm zap
    /dev/vg_nvme/lv_4'

[6465357]  rados:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final}
-----------------------------------------------------------------
time:   00:19:32
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465357/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465357/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fb654ea6e4d24cf4ac1f467adde82890

    reached maximum tries (180) after waiting for 180 seconds

[6465358]  rados:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root}
-----------------------------------------------------------------
time:   00:21:33
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465358/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465358/
sentry: https://sentry.ceph.com/organizations/ceph/?query=edd3706f8798425e868d2890fbbe7fb7

    Command failed on smithi204 with status 5: 'sudo systemctl stop
    ceph-1f63c4a0-3833-11ec-8c28-001a4aab830c@mon.b'

[6465360]  rados:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:22:01
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465360/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465360/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dd6c45d5f3444c32b7cf998bd26ba909

    Command failed on smithi197 with status 5: 'sudo systemctl stop
    ceph-201394ca-3833-11ec-8c28-001a4aab830c@mon.b'

[6465361]  rados:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:22:48
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465361/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465361/
sentry: https://sentry.ceph.com/organizations/ceph/?query=325d38b6788e4817b3a9fb2e40996fb2

    Command failed on smithi098 with status 5: 'sudo systemctl stop
    ceph-511c043a-3833-11ec-8c28-001a4aab830c@mon.b'

[6465362]  rados:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw 3-final}
-----------------------------------------------------------------
time:   00:20:58
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465362/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465362/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e5f27fdc78ca4d0db7326789dda0c5f5

    reached maximum tries (180) after waiting for 180 seconds

[6465363]  rados:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   01:14:23
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465363/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465363/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a6e240a9feb04ace879fbc78c8c3eb16

    reached maximum tries (180) after waiting for 180 seconds

[6465364]  rados:cephadm/smoke/{0-nvme-loop agent/off distro/ubuntu_20.04 fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:23:57
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465364/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465364/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7c4b5382f335413a946a3ad396787b75

    Command failed on smithi178 with status 5: 'sudo systemctl stop
    ceph-d2c786ea-3832-11ec-8c28-001a4aab830c@mon.b'

[6465365]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:32:49
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465365/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465365/
sentry: https://sentry.ceph.com/organizations/ceph/?query=36bf45f851f146e0beb683f87e191866

    Command failed on smithi172 with status 5: 'sudo systemctl stop
    ceph-5bcb2018-3834-11ec-8c28-001a4aab830c@mon.b'

[6465366]  rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/off mon_election/connectivity task/test_orch_cli}
-----------------------------------------------------------------
time:   00:10:41
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465366/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465366/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6f87ff15b5794ab085b5e95c32abbe2d

    Command failed on smithi074 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:434b1409634993c5a89bbdcd2c0af0f073acdf91 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    28f9d6bc-3833-11ec-8c28-001a4aab830c -- ceph-volume lvm zap
    /dev/vg_nvme/lv_4'

[6465367]  rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:34:51
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465367/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465367/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2b5391249fc44b27afca91001e454b22

    Command failed on smithi041 with status 5: 'sudo systemctl stop
    ceph-8fb44a4e-3834-11ec-8c28-001a4aab830c@mon.b'



Dead
=================================================================
[6465273]  rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465273/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465273/

    hit max job timeout

[6465274]  rados:cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465274/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465274/

    hit max job timeout

[6465280]  rados:cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465280/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465280/

    hit max job timeout

[6465294]  rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465294/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465294/

    hit max job timeout

[6465302]  rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465302/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465302/

    hit max job timeout

[6465309]  rados:cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465309/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465309/

    hit max job timeout

[6465311]  rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465311/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465311/

    hit max job timeout

[6465322]  rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465322/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465322/

    hit max job timeout

[6465328]  rados:cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465328/
log:    http://qa-proxy.ceph.com/teuthology/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465328/

    hit max job timeout



Running
=================================================================
[6465340] rados:cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final}
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465340/

[6465342] rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic}
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465342/

[6465351] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465351/

[6465359] rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465359/



Pass
=================================================================
[6465275] rados:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.3_container_tools_3.0} 2-node-mgr agent/off orchestrator_cli}
time:   00:16:10
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465275/

[6465282] rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/on mon_election/classic task/test_adoption}
time:   00:14:03
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465282/

[6465299] rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/on mon_election/classic task/test_cephadm_repos}
time:   00:07:34
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465299/

[6465323] rados:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.3_container_tools_3.0} 2-node-mgr agent/on orchestrator_cli}
time:   00:16:24
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465323/

[6465330] rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/off mon_election/connectivity task/test_adoption}
time:   00:15:10
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465330/

[6465348] rados:cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 agent/off mon_election/connectivity task/test_cephadm_repos}
time:   00:07:07
info:   http://pulpito.ceph.com/adking-2021-10-28_20:09:21-rados:cephadm-wip-adk-testing-2021-10-28-1305-distro-basic-smithi/6465348/