2023-06-01T08:40:13.402 INFO:root:teuthology version: 0.0.1.dev116+g62814ab
2023-06-01T08:40:13.403 INFO:teuthology.results:subset: '1/14'
2023-06-01T08:40:13.403 INFO:teuthology.results:seed: '5609'
2023-06-01T09:41:50.342 INFO:root:teuthology version: 0.0.1.dev116+g62814ab
2023-06-01T09:41:50.397 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-06-01T10:21:53.959 INFO:teuthology.results:Tests finished! gathering results...
2023-06-01T10:21:58.233 INFO:scrape:Found 92 jobs
2023-06-01T10:22:11.930 INFO:scrape:Found 4 distinct failure reasons
2023-06-01T10:22:11.931 INFO:scrape:Failure: Command failed on smithi130 with status 5: 'sudo systemctl stop ceph-47a3e27e-005f-11ee-9b1d-001a4aab830c@mon.smithi130'
2023-06-01T10:22:11.931 INFO:scrape:66 jobs
2023-06-01T10:22:11.932 INFO:scrape:suites intersection: []
2023-06-01T10:22:11.932 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/repave-all}', '2-ops/rm-zap-add}', '2-ops/rm-zap-flag}', '2-ops/rm-zap-wait}', '2-ops/rmdir-reactivate}', '2-services/basic', '2-services/client-keyring', '2-services/iscsi', '2-services/jaeger', '2-services/mirror', '2-services/nfs', '2-services/nfs-haproxy-proto', '2-services/nfs-ingress', '2-services/nfs-ingress-rgw-bucket', '2-services/nfs-ingress-rgw-user', '2-services/nfs-ingress2', '2-services/nfs-keepalive-only', '2-services/nfs2', '2-services/rgw', '2-services/rgw-ingress', '2-thrash', '3-final}', '3-tasks/rados_api_tests', '3-tasks/radosbench', '3-tasks/small-objects', '3-tasks/snaps-few-objects', 'agent/off', 'agent/on', 'base/install', 'cluster/{fixed-3', 'fixed-2', 'mode/packaged', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'openstack}', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/osds/{0-distro/ubuntu_20.04', 'orch:cephadm/rbd_iscsi/{0-single-container-host', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04', 'orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/smoke/{0-distro/ubuntu_20.04', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/thrash/{0-distro/ubuntu_20.04', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/with-work/{0-distro/ubuntu_20.04', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/workunits/{0-distro/ubuntu_20.04', 'root}', 'start', 'start}', 'task/test_orch_cli_mon}', 'task/test_rgw_multisite}', 'task/test_set_mon_crush_locations}', 'tasks/rados_api_tests}', 'tasks/rados_python}', 'tasks/rotate-keys}', 'workloads/cephadm_iscsi}']
2023-06-01T10:22:11.933 INFO:scrape:
2023-06-01T10:22:11.933 INFO:scrape:Failure: Command failed on smithi173 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9691a508-0062-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd smithi173:vg_nvme/lv_4'
2023-06-01T10:22:11.933 INFO:scrape:8 jobs: ['7293365', '7293295', '7293357', '7293292', '7293337', '7293300', '7293351', '7293285']
2023-06-01T10:22:11.934 INFO:scrape:suites intersection: []
2023-06-01T10:22:11.934 INFO:scrape:suites union: ['1-start', '2-services/basic', '2-services/rgw', '3-final}', 'agent/off', 'agent/on', 'mon_election/classic', 'mon_election/connectivity', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04}', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_iscsi_container/{centos_8.stream_container_tools', 'task/test_nfs}', 'task/test_orch_cli}', 'test_iscsi_container}}']
2023-06-01T10:22:11.934 INFO:scrape:
2023-06-01T10:22:11.935 INFO:scrape:Failure: Command failed on smithi055 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f5a318ca-0061-11ee-9b1d-001a4aab830c -e sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | length == 1\'"\'"\'\''
2023-06-01T10:22:11.935 INFO:scrape:13 jobs: ['7293347', '7293323', '7293340', '7293289', '7293305', '7293319', '7293301', '7293373', '7293359', '7293320', '7293362', '7293332', '7293352']
2023-06-01T10:22:11.935 INFO:scrape:suites intersection: []
2023-06-01T10:22:11.936 INFO:scrape:suites union: ['1-bootstrap/16.2.0', '1-bootstrap/16.2.4', '1-bootstrap/16.2.5', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-inline/no', '3-inline/yes', '3-upgrade-mgr-staggered', '3-upgrade-with-workload', '3-upgrade/simple', '3-upgrade/staggered', '4-config-upgrade/{fail_fs}', '4-final}', '4-verify}', '4-wait', '5-upgrade-ls', '5-upgrade-with-workload', '6-verify}}', 'agent/off', 'agent/on', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/no', 'fail_fs/yes', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon_election/classic}', 'mon_election/connectivity}', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4']
2023-06-01T10:22:11.936 INFO:scrape:
2023-06-01T10:22:11.936 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi153 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7519db486cb7415b3ba2be707112c97a00f09b37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'
2023-06-01T10:22:11.937 INFO:scrape:1 jobs: ['7293334']
2023-06-01T10:22:11.937 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'orch:cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_cephadm}']
2023-06-01T10:22:11.937 INFO:scrape:
2023-06-01T10:22:11.941 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/
failed:  88
dead:    0
running: 0
waiting: 0
queued:  0
passed:  4


Fail
=================================================================
[7293285]  orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
-----------------------------------------------------------------
time:   00:17:00
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293285/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293285/
sentry: https://sentry.ceph.com/organizations/ceph/?query=79567331d3e4451b88abb8e83db32005

    Command failed on smithi046 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    e5e2ad34-005b-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd
    smithi046:vg_nvme/lv_4'

[7293286]  orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag}
-----------------------------------------------------------------
time:   00:20:07
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293286/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293286/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7df4de70cb174d6f9ffbbf1588bd962e

    Command failed on smithi194 with status 5: 'sudo systemctl stop
    ceph-107a1416-005b-11ee-9b1d-001a4aab830c@mon.smithi194'

[7293287]  orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:22:24
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293287/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293287/
sentry: https://sentry.ceph.com/organizations/ceph/?query=88760c01f62a4bc9af398298a99233f7

    Command failed on smithi079 with status 5: 'sudo systemctl stop
    ceph-2ebc80ea-005a-11ee-9b1d-001a4aab830c@mon.b'

[7293288]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
-----------------------------------------------------------------
time:   00:20:10
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293288/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293288/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d6b9f13898214f43898d1a3753f9c0e6

    Command failed on smithi139 with status 5: 'sudo systemctl stop
    ceph-17be0fb6-005b-11ee-9b1d-001a4aab830c@mon.smithi139'

[7293289]  orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:22:10
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293289/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293289/
sentry: https://sentry.ceph.com/organizations/ceph/?query=367b5673f2b44912a2f7c73d43fa19e4

    Command failed on smithi008 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    1d2dfd76-005b-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293291]  orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi}
-----------------------------------------------------------------
time:   00:19:35
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293291/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293291/
sentry: https://sentry.ceph.com/organizations/ceph/?query=759a643594ba4cc08ffa1a6129ae74c5

    Command failed on smithi083 with status 5: 'sudo systemctl stop
    ceph-5bb75ede-005b-11ee-9b1d-001a4aab830c@mon.b'

[7293292]  orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   00:11:18
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293292/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293292/
sentry: https://sentry.ceph.com/organizations/ceph/?query=29e26780435140cf8be2973178f6eacc

    Command failed on smithi141 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    92b43eda-005a-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd
    smithi141:vg_nvme/lv_4'

[7293293]  orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:19:11
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293293/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293293/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b54e87660d634c99b947ff0a25323dd1

    Command failed on smithi153 with status 5: 'sudo systemctl stop
    ceph-2d9fbea6-005b-11ee-9b1d-001a4aab830c@mon.b'

[7293294]  orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:32:35
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293294/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293294/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a30b16c5bca24d1d9d5f94fa18982e63

    Command failed on smithi193 with status 5: 'sudo systemctl stop
    ceph-433bb656-005c-11ee-9b1d-001a4aab830c@mon.b'

[7293295]  orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_nfs}
-----------------------------------------------------------------
time:   00:15:12
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293295/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293295/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cebbc698f128478c81b0eae6098eca15

    Command failed on smithi156 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    5d6b1c7a-005b-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd
    smithi156:vg_nvme/lv_4'

[7293296]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
-----------------------------------------------------------------
time:   00:19:08
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293296/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293296/
sentry: https://sentry.ceph.com/organizations/ceph/?query=16375e22f1c0474587e1644ef6c7266e

    Command failed on smithi135 with status 5: 'sudo systemctl stop
    ceph-4f7adcd6-005b-11ee-9b1d-001a4aab830c@mon.smithi135'

[7293297]  orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:29:40
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293297/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293297/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a1e9a5d1e64c4a9d8a69d98030144994

    Command failed on smithi146 with status 5: 'sudo systemctl stop
    ceph-80692cee-005b-11ee-9b1d-001a4aab830c@mon.b'

[7293298]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
-----------------------------------------------------------------
time:   00:22:27
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293298/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293298/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a4f097b177704620a07582febd0296b5

    Command failed on smithi130 with status 5: 'sudo systemctl stop
    ceph-20df73e6-005b-11ee-9b1d-001a4aab830c@mon.smithi130'

[7293299]  orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys}
-----------------------------------------------------------------
time:   00:28:22
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293299/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293299/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bfa7af7ed43f4aa4a028ea36fa85a430

    Command failed on smithi150 with status 5: 'sudo systemctl stop
    ceph-d6027df4-005b-11ee-9b1d-001a4aab830c@mon.b'

[7293300]  orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli}
-----------------------------------------------------------------
time:   00:11:56
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293300/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293300/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c892aa985e394ca5a48901de64b54c02

    Command failed on smithi072 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    f582dc5a-005b-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd
    smithi072:vg_nvme/lv_4'

[7293301]  orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity}
-----------------------------------------------------------------
time:   00:24:48
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293301/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293301/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d6a1f4f8b69a4b05a10a0fb7a677751f

    Command failed on smithi007 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    6721ef14-005b-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293302]  orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:24:21
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293302/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293302/
sentry: https://sentry.ceph.com/organizations/ceph/?query=33259fd92a9f43068aa01b848ac173ae

    Command failed on smithi165 with status 5: 'sudo systemctl stop ceph-
    ba411f58-005b-11ee-9b1d-001a4aab830c@mon.smithi165'

[7293303]  orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:21:12
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293303/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293303/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3a6959af89924faeb52fd4481a49b697

    Command failed on smithi163 with status 5: 'sudo systemctl stop
    ceph-6fea0370-005b-11ee-9b1d-001a4aab830c@mon.b'

[7293304]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
-----------------------------------------------------------------
time:   00:21:47
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293304/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293304/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ae4e5021b4c54088abbe2ff8c248729d

    Command failed on smithi169 with status 5: 'sudo systemctl stop
    ceph-75f6b498-005b-11ee-9b1d-001a4aab830c@mon.smithi169'

[7293305]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:28:36
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293305/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293305/
sentry: https://sentry.ceph.com/organizations/ceph/?query=14a7a6509347406d9bcd75dda1e721b1

    Command failed on smithi155 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    28bb10ae-005e-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293306]  orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
-----------------------------------------------------------------
time:   00:26:29
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293306/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293306/
sentry: https://sentry.ceph.com/organizations/ceph/?query=aac0b4ffc3974f3383d365524271550e

    Command failed on smithi148 with status 5: 'sudo systemctl stop
    ceph-139522f2-005c-11ee-9b1d-001a4aab830c@mon.smithi148'

[7293307]  orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:23:29
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293307/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293307/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f537f5f4c2ff4ee3ba775412c82f3d01

    Command failed on smithi136 with status 5: 'sudo systemctl stop
    ceph-b78ec11e-005d-11ee-9b1d-001a4aab830c@mon.b'

[7293308]  orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:24:14
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293308/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293308/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1b23eda3a60240289df5bdba6cdd4312

    Command failed on smithi142 with status 5: 'sudo systemctl stop
    ceph-b12fb872-005e-11ee-9b1d-001a4aab830c@mon.b'

[7293309]  orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_orch_cli_mon}
-----------------------------------------------------------------
time:   00:26:19
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293309/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293309/
sentry: https://sentry.ceph.com/organizations/ceph/?query=67341cb5fb254573947c0bdb25c675f7

    Command failed on smithi078 with status 5: 'sudo systemctl stop
    ceph-6993f636-005e-11ee-9b1d-001a4aab830c@mon.b'

[7293310]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final}
-----------------------------------------------------------------
time:   00:19:05
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293310/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293310/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5b0382ffe46643e7b3a4fe92f3764ca0

    Command failed on smithi085 with status 5: 'sudo systemctl stop
    ceph-8b0282f2-005d-11ee-9b1d-001a4aab830c@mon.smithi085'

[7293311]  orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:31:16
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293311/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293311/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d830be89e0844c12a879271846bf6a26

    Command failed on smithi188 with status 5: 'sudo systemctl stop
    ceph-f59a71fe-005f-11ee-9b1d-001a4aab830c@mon.b'

[7293312]  orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_rgw_multisite}
-----------------------------------------------------------------
time:   00:27:12
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293312/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293312/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1de90c17d7644ec389d871fd1f44b964

    Command failed on smithi097 with status 5: 'sudo systemctl stop
    ceph-3c28c77a-005f-11ee-9b1d-001a4aab830c@mon.b'

[7293313]  orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:19:41
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293313/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293313/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f22cc2f705cd4c3aad3f19dbcadc71a9

    Command failed on smithi131 with status 5: 'sudo systemctl stop
    ceph-582c5a50-005e-11ee-9b1d-001a4aab830c@mon.b'

[7293314]  orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
-----------------------------------------------------------------
time:   00:25:12
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293314/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293314/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4959743734364552a92ed48f4340f6bb

    Command failed on smithi178 with status 5: 'sudo systemctl stop
    ceph-8a455b32-005d-11ee-9b1d-001a4aab830c@mon.smithi178'

[7293315]  orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:26:53
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293315/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293315/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a31f3b099c004aea93dd628eadaa9d23

    Command failed on smithi121 with status 5: 'sudo systemctl stop
    ceph-6f620932-005d-11ee-9b1d-001a4aab830c@mon.b'

[7293316]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs2 3-final}
-----------------------------------------------------------------
time:   00:19:11
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293316/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293316/
sentry: https://sentry.ceph.com/organizations/ceph/?query=64145acbfd834527a19322080983cea9

    Command failed on smithi173 with status 5: 'sudo systemctl stop
    ceph-084e1578-005e-11ee-9b1d-001a4aab830c@mon.smithi173'

[7293317]  orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root}
-----------------------------------------------------------------
time:   00:31:00
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293317/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293317/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5cf238ec93474d2a818260427bd21583

    Command failed on smithi174 with status 5: 'sudo systemctl stop ceph-
    babbd794-005f-11ee-9b1d-001a4aab830c@mon.b'

[7293318]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
-----------------------------------------------------------------
time:   00:23:51
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293318/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293318/
sentry: https://sentry.ceph.com/organizations/ceph/?query=285508e67bbf4a029112c0a095418dc1

    Command failed on smithi202 with status 5: 'sudo systemctl stop
    ceph-b2c7c4be-005d-11ee-9b1d-001a4aab830c@mon.smithi202'

[7293319]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:34:41
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293319/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293319/
sentry: https://sentry.ceph.com/organizations/ceph/?query=73ed60d8b8864e6ba1822a2fbda00951

    Command failed on smithi031 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    45922602-0060-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.mgr | length == 1\'"\'"\'\''

[7293320]  orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic}
-----------------------------------------------------------------
time:   00:31:54
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293320/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293320/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8109bc2f4e534c59b5d7ab9274dcb93e

    Command failed on smithi005 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    78f6db20-005e-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.mgr | length == 1\'"\'"\'\''

[7293321]  orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys}
-----------------------------------------------------------------
time:   00:29:57
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293321/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293321/
sentry: https://sentry.ceph.com/organizations/ceph/?query=246ce61145914d9fb6a93594c1ceec63

    Command failed on smithi114 with status 5: 'sudo systemctl stop
    ceph-1181622a-005f-11ee-9b1d-001a4aab830c@mon.b'

[7293322]  orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_set_mon_crush_locations}
-----------------------------------------------------------------
time:   00:28:45
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293322/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293322/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7d070323d23647adb48b674b1885b612

    Command failed on smithi023 with status 5: 'sudo systemctl stop ceph-
    fc0e05ce-005e-11ee-9b1d-001a4aab830c@mon.b'

[7293323]  orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:18:30
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293323/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293323/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fec2126814474b15a1cc8d53719cd058

    Command failed on smithi033 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    5db71db0-005f-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293324]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final}
-----------------------------------------------------------------
time:   00:21:39
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293324/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293324/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1cedce2cb6bb425bb1720181a8996ed7

    Command failed on smithi079 with status 5: 'sudo systemctl stop
    ceph-26b60480-005e-11ee-9b1d-001a4aab830c@mon.smithi079'

[7293325]  orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all}
-----------------------------------------------------------------
time:   00:22:12
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293325/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293325/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9780b70c070b4e8182cea5cb051a00f9

    Command failed on smithi107 with status 5: 'sudo systemctl stop
    ceph-68eba6a2-005e-11ee-9b1d-001a4aab830c@mon.smithi107'

[7293326]  orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:19:38
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293326/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293326/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5ec5b4159d2e40c4b78f1432e285f9b3

    Command failed on smithi124 with status 5: 'sudo systemctl stop
    ceph-7de2c8aa-005f-11ee-9b1d-001a4aab830c@mon.b'

[7293327]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   00:22:05
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293327/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293327/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2a92e20c930c41b5ac6e902cf98620f8

    Command failed on smithi203 with status 5: 'sudo systemctl stop
    ceph-9f32d5b4-005e-11ee-9b1d-001a4aab830c@mon.smithi203'

[7293328]  orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:28:36
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293328/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293328/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ae1ccab77079439fa414ab884bd65145

    Command failed on smithi081 with status 5: 'sudo systemctl stop
    ceph-642a9140-005f-11ee-9b1d-001a4aab830c@mon.b'

[7293329]  orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:26:34
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293329/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293329/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dcff0db7259942a9ba2c32551b2fb9fd

    Command failed on smithi152 with status 5: 'sudo systemctl stop
    ceph-6fad1e98-005f-11ee-9b1d-001a4aab830c@mon.b'

[7293331]  orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final}
-----------------------------------------------------------------
time:   00:27:01
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293331/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293331/
sentry: https://sentry.ceph.com/organizations/ceph/?query=64aab918600b4f3ba0825e1848b9258e

    Command failed on smithi102 with status 5: 'sudo systemctl stop
    ceph-e65eadaa-005e-11ee-9b1d-001a4aab830c@mon.smithi102'

[7293332]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:31:47
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293332/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293332/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6ad97a79c04242f3ac5a0af3662081ea

    Command failed on smithi022 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    67388430-0061-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293333]  orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:28:40
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293333/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293333/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6db76593135b4fffae460c09655fa279

    Command failed on smithi067 with status 5: 'sudo systemctl stop
    ceph-6c168cd8-005f-11ee-9b1d-001a4aab830c@mon.b'

[7293334]  orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm}
-----------------------------------------------------------------
time:   00:12:39
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293334/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293334/
sentry: https://sentry.ceph.com/organizations/ceph/?query=62c27bb8d0294406a597e5efc459b6f2

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi153 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=7519db486cb7415b3ba2be707112c97a00f09b37
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[7293335]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final}
-----------------------------------------------------------------
time:   00:19:56
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293335/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293335/
sentry: https://sentry.ceph.com/organizations/ceph/?query=76e25ca56b964cf7b3ba189b3c29db81

    Command failed on smithi195 with status 5: 'sudo systemctl stop ceph-
    cd01a7ee-005f-11ee-9b1d-001a4aab830c@mon.smithi195'

[7293337]  orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final}
-----------------------------------------------------------------
time:   00:09:37
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293337/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293337/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4d2c9ccb31cf4520bced9a1bce65d067

    Command failed on smithi135 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    2944b312-005f-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd
    smithi135:vg_nvme/lv_4'

[7293338]  orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:19:26
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293338/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293338/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7ff63248db4f4349ab6f42446482fc59

    Command failed on smithi105 with status 5: 'sudo systemctl stop
    ceph-015d0592-0060-11ee-9b1d-001a4aab830c@mon.b'

[7293339]  orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:28:11
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293339/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293339/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b5ee7d1e3b14c86ad6a6817a10d5d3d

    Command failed on smithi154 with status 5: 'sudo systemctl stop
    ceph-1e748b00-0060-11ee-9b1d-001a4aab830c@mon.b'

[7293340]  orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
-----------------------------------------------------------------
time:   00:22:49
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293340/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293340/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7e0079743d1542d4b9696356a8ccd0cc

    Command failed on smithi084 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    4ed36178-005f-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293341]  orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:26:35
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293341/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293341/
sentry: https://sentry.ceph.com/organizations/ceph/?query=be2edbafaa6e431f8dbeee2531b821d0

    Command failed on smithi130 with status 5: 'sudo systemctl stop
    ceph-47a3e27e-005f-11ee-9b1d-001a4aab830c@mon.smithi130'

[7293342]  orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:18:36
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293342/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293342/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f47dd2b93ff94fcaae9a10f111196461

    Command failed on smithi163 with status 5: 'sudo systemctl stop
    ceph-31c97fc6-0060-11ee-9b1d-001a4aab830c@mon.b'

[7293343]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final}
-----------------------------------------------------------------
time:   00:18:34
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293343/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293343/
sentry: https://sentry.ceph.com/organizations/ceph/?query=06164b776615455490e160be101ab0a2

    Command failed on smithi138 with status 5: 'sudo systemctl stop
    ceph-f836185a-005f-11ee-9b1d-001a4aab830c@mon.smithi138'

[7293344]  orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys}
-----------------------------------------------------------------
time:   00:23:33
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293344/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293344/
sentry: https://sentry.ceph.com/organizations/ceph/?query=038a65767ef646a5920bc9c6b68a3777

    Command failed on smithi159 with status 5: 'sudo systemctl stop
    ceph-0440781a-0061-11ee-9b1d-001a4aab830c@mon.b'

[7293346]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final}
-----------------------------------------------------------------
time:   00:23:46
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293346/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293346/
sentry: https://sentry.ceph.com/organizations/ceph/?query=67c372ac83574707890c0cbfba6c6605

    Command failed on smithi119 with status 5: 'sudo systemctl stop ceph-
    bac0a58a-005f-11ee-9b1d-001a4aab830c@mon.smithi119'

[7293347]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:31:04
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293347/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293347/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9cc11b47fe0046e5a4f2084d837d4514

    Command failed on smithi055 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    f5a318ca-0061-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.mgr | length == 1\'"\'"\'\''

[7293348]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final}
-----------------------------------------------------------------
time:   00:22:26
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293348/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293348/
sentry: https://sentry.ceph.com/organizations/ceph/?query=20cee1b7589244869ef0973a5cdd5d05

    Command failed on smithi177 with status 5: 'sudo systemctl stop ceph-
    bee9d33e-005f-11ee-9b1d-001a4aab830c@mon.smithi177'

[7293349]  orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root}
-----------------------------------------------------------------
time:   00:29:16
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293349/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293349/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b03a20d1331e4c59914b47e1c1bef5d3

    Command failed on smithi165 with status 5: 'sudo systemctl stop
    ceph-2c42f9e2-0060-11ee-9b1d-001a4aab830c@mon.b'

[7293350]  orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:22:02
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293350/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293350/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dd910de385de4396a4188aaa9f7f392a

    Command failed on smithi167 with status 5: 'sudo systemctl stop
    ceph-941ea6a0-0061-11ee-9b1d-001a4aab830c@mon.b'

[7293351]  orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
-----------------------------------------------------------------
time:   00:16:20
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293351/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293351/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4753eda1688b488f95cb00deef1fd730

    Command failed on smithi193 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    e4c1ce3e-0061-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd
    smithi193:vg_nvme/lv_4'

[7293352]  orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:18:09
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293352/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293352/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1c52cecd9c6741b59f5396a80610b9f0

    Command failed on smithi123 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.5 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    02c56630-0061-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293353]  orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
-----------------------------------------------------------------
time:   00:20:06
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293353/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293353/
sentry: https://sentry.ceph.com/organizations/ceph/?query=10ecd65144ae4c62af5bed889452198e

    Command failed on smithi151 with status 5: 'sudo systemctl stop
    ceph-0613fc52-0061-11ee-9b1d-001a4aab830c@mon.smithi151'

[7293354]  orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:22:27
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293354/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293354/
sentry: https://sentry.ceph.com/organizations/ceph/?query=742671742d3a47d197a437ac99a18e27

    Command failed on smithi170 with status 5: 'sudo systemctl stop
    ceph-6aec975c-0060-11ee-9b1d-001a4aab830c@mon.b'

[7293355]  orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
-----------------------------------------------------------------
time:   00:26:54
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293355/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293355/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ad6862b9229743dcadecd2338cb463d2

    Command failed on smithi149 with status 5: 'sudo systemctl stop
    ceph-78a15450-0060-11ee-9b1d-001a4aab830c@mon.smithi149'

[7293356]  orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:29:02
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293356/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293356/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ea7445495c284c9b914afe6aeae40d4d

    Command failed on smithi146 with status 5: 'sudo systemctl stop
    ceph-86104c94-0061-11ee-9b1d-001a4aab830c@mon.b'

[7293357]  orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_nfs}
-----------------------------------------------------------------
time:   00:20:26
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293357/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293357/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b09af9f86ff41d0aede8ee2d9af4d9e

    Command failed on smithi039 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    23960f4e-0062-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd
    smithi039:vg_nvme/lv_4'

[7293358]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
-----------------------------------------------------------------
time:   00:20:05
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293358/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293358/
sentry: https://sentry.ceph.com/organizations/ceph/?query=78f9b7020fe64eb1b6f93d0d10403ede

    Command failed on smithi182 with status 5: 'sudo systemctl stop
    ceph-1aeeebd6-0062-11ee-9b1d-001a4aab830c@mon.smithi182'

[7293359]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:25:33
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293359/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293359/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f462156d68e94abea13d1ecafdf25102

    Command failed on smithi120 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    ccc70c80-0062-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293360]  orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:19:53
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293360/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293360/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5451b1d7a7154857b40ac459c379391b

    Command failed on smithi148 with status 5: 'sudo systemctl stop
    ceph-81e1d9ca-0062-11ee-9b1d-001a4aab830c@mon.b'

[7293361]  orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:23:54
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293361/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293361/
sentry: https://sentry.ceph.com/organizations/ceph/?query=219361da812b4f528c763061279ef2f7

    Command failed on smithi085 with status 5: 'sudo systemctl stop
    ceph-b78b0f9c-0062-11ee-9b1d-001a4aab830c@mon.b'

[7293362]  orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic}
-----------------------------------------------------------------
time:   00:20:15
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293362/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293362/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9dff9a0ac1744d4ea567ca2249319d3d

    Command failed on smithi141 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    15462118-0062-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7293363]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
-----------------------------------------------------------------
time:   00:20:00
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293363/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293363/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eb23025e56574272b80fae291db4ef9b

    Command failed on smithi136 with status 5: 'sudo systemctl stop
    ceph-4396bfdc-0062-11ee-9b1d-001a4aab830c@mon.smithi136'

[7293364]  orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys}
-----------------------------------------------------------------
time:   00:36:06
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293364/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293364/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fb6efa362ca84dadaaaee7ac72083859

    Command failed on smithi202 with status 5: 'sudo systemctl stop
    ceph-b8488620-0063-11ee-9b1d-001a4aab830c@mon.b'

[7293365]  orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli}
-----------------------------------------------------------------
time:   00:16:48
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293365/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293365/
sentry: https://sentry.ceph.com/organizations/ceph/?query=31c10cf9be5b46edabe92be8672c3782

    Command failed on smithi173 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:7519db486cb7415b3ba2be707112c97a00f09b37 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    9691a508-0062-11ee-9b1d-001a4aab830c -- ceph orch daemon add osd
    smithi173:vg_nvme/lv_4'

[7293366]  orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:19:52
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293366/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293366/
sentry: https://sentry.ceph.com/organizations/ceph/?query=25dc91fcf1b5465b8727992712f3722e

    Command failed on smithi082 with status 5: 'sudo systemctl stop ceph-
    eab566a6-0062-11ee-9b1d-001a4aab830c@mon.smithi082'

[7293367]  orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:20:30
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293367/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293367/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7d5c26f1a0b6446896cb76a2acce8a60

    Command failed on smithi135 with status 5: 'sudo systemctl stop
    ceph-c9013d24-0061-11ee-9b1d-001a4aab830c@mon.b'

[7293368]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
-----------------------------------------------------------------
time:   00:22:03
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293368/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293368/
sentry: https://sentry.ceph.com/organizations/ceph/?query=865415bd644646a6acaca15cfda7c110

    Command failed on smithi155 with status 5: 'sudo systemctl stop
    ceph-056ab56a-0062-11ee-9b1d-001a4aab830c@mon.smithi155'

[7293369]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
-----------------------------------------------------------------
time:   00:21:02
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293369/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293369/
sentry: https://sentry.ceph.com/organizations/ceph/?query=05a25342dce14c089695deb31f2874f9

    Command failed on smithi184 with status 5: 'sudo systemctl stop
    ceph-11bb1b66-0062-11ee-9b1d-001a4aab830c@mon.smithi184'

[7293370]  orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:30:39
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293370/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293370/
sentry: https://sentry.ceph.com/organizations/ceph/?query=54aef0fe1f4747d9bbcddb9ec88ea180

    Command failed on smithi187 with status 5: 'sudo systemctl stop
    ceph-7c680bac-0064-11ee-9b1d-001a4aab830c@mon.b'

[7293371]  orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:28:23
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293371/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293371/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e330b148d66d44b69e25f63b7f8a0f4a

    Command failed on smithi160 with status 5: 'sudo systemctl stop
    ceph-a2f797ee-0062-11ee-9b1d-001a4aab830c@mon.b'

[7293372]  orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli_mon}
-----------------------------------------------------------------
time:   00:30:38
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293372/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293372/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5e417eb9a95d455a9e9b831e1a15bb18

    Command failed on smithi073 with status 5: 'sudo systemctl stop
    ceph-d5f8562e-0062-11ee-9b1d-001a4aab830c@mon.b'

[7293373]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:26:35
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293373/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293373/
sentry: https://sentry.ceph.com/organizations/ceph/?query=be062ab3ca6f404c9490f9a5d3517344

    Command failed on smithi070 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    b385190a-0063-11ee-9b1d-001a4aab830c -e
    sha1=7519db486cb7415b3ba2be707112c97a00f09b37 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.mgr | length == 1\'"\'"\'\''

[7293374]  orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final}
-----------------------------------------------------------------
time:   00:27:11
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293374/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293374/
sentry: https://sentry.ceph.com/organizations/ceph/?query=529bc7e506354de180882cdf60453c6f

    Command failed on smithi162 with status 5: 'sudo systemctl stop ceph-
    ab2ae632-0062-11ee-9b1d-001a4aab830c@mon.smithi162'

[7293375]  orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:25:18
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293375/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293375/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fec5aa0f2be64a4caeb88ac3cfaa6a48

    Command failed on smithi078 with status 5: 'sudo systemctl stop
    ceph-1b37c17e-0064-11ee-9b1d-001a4aab830c@mon.b'

[7293376]  orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_rgw_multisite}
-----------------------------------------------------------------
time:   00:24:59
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293376/
log:    http://qa-proxy.ceph.com/teuthology/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293376/
sentry: https://sentry.ceph.com/organizations/ceph/?query=60c05df72ff949779618219de8099e39

    Command failed on smithi107 with status 5: 'sudo systemctl stop
    ceph-5c7fe4ae-0064-11ee-9b1d-001a4aab830c@mon.b'



Pass
=================================================================
[7293290] orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/off orchestrator_cli}
time:   00:18:39
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293290/

[7293330] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_adoption}
time:   00:16:06
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293330/

[7293336] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_3.0} 2-node-mgr agent/on orchestrator_cli}
time:   00:25:54
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293336/

[7293345] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_cephadm_repos}
time:   00:09:38
info:   http://pulpito.ceph.com/adking-2023-06-01_03:43:37-orch:cephadm-wip-adk4-testing-2023-05-31-2041-distro-default-smithi/7293345/