2022-02-04T07:40:29.117 INFO:root:teuthology version: 1.1.0-3094160c
2022-02-04T07:40:29.120 INFO:teuthology.results:subset: '1111/9999'
2022-02-04T07:40:29.120 INFO:teuthology.results:seed: '1489'
2022-02-04T08:11:27.181 INFO:root:teuthology version: 1.1.0-3094160c
2022-02-04T08:11:27.186 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-02-04T08:44:29.842 INFO:teuthology.results:Tests finished! gathering results...
2022-02-04T08:44:33.026 INFO:scrape:Found 75 jobs
2022-02-04T08:44:34.354 INFO:scrape:Found 4 distinct failure reasons
2022-02-04T08:44:34.355 INFO:scrape:Failure: no results found at https://shaman.ceph.com/api/search/?project=ceph&distros=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3342a92766
2022-02-04T08:44:34.356 INFO:scrape:12 jobs: ['6661143', '6661155', '6661135', '6661179', '6661163', '6661198', '6661170', '6661189', '6661134', '6661173', '6661180', '6661182']
2022-02-04T08:44:34.356 INFO:scrape:suites intersection: []
2022-02-04T08:44:34.357 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/repave-all}', '2-ops/rm-zap-add}', '2-ops/rm-zap-flag}', '2-services/iscsi', '2-services/mirror', '2-services/nfs', '2-services/nfs-ingress2', '2-thrash', '3-final}', '3-tasks/small-objects', '3-tasks/snaps-few-objects', 'agent/off', 'agent/on', 'fixed-2', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async', 'msgr/async-v1only', 'orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8', 'root}', 'start', 'start}', 'tasks/rados_api_tests}']
2022-02-04T08:44:34.357 INFO:scrape:
2022-02-04T08:44:34.358 INFO:scrape:Failure: Command failed on smithi083 with status 1: 'sudo yum install -y kernel'
2022-02-04T08:44:34.358 INFO:scrape:48 jobs
2022-02-04T08:44:34.359 INFO:scrape:suites intersection: []
2022-02-04T08:44:34.359 INFO:scrape:suites union: ['0-nvme-loop', '1-bootstrap/16.2.4', '1-bootstrap/16.2.5', '1-bootstrap/octopus', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-node-mgr', '2-ops/repave-all}', '2-ops/rm-zap-flag}', '2-ops/rm-zap-wait}', '2-ops/rmdir-reactivate}', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '2-services/basic', '2-services/client-keyring', '2-services/nfs-ingress', '2-services/nfs-ingress-rgw-user', '2-services/nfs-ingress2', '2-services/rgw', '2-services/rgw-ingress', '2-thrash', '3-final}', '3-start-upgrade', '3-tasks/rados_api_tests', '3-tasks/radosbench', '3-tasks/small-objects', '3-tasks/snaps-few-objects', '3-upgrade-with-workload', '3-verify}', '4-final}', '4-verify}}', '4-wait', '5-upgrade-ls', 'agent/off', 'agent/on', 'base/install', 'centos_8.stream_container_tools', 'cluster/{fixed-3', 'conf/{client', 'fixed-2', 'mds', 'mode/packaged', 'mode/root', 'mon', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'openstack}', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.2_container_tools_3.0}', 'orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'orch:cephadm/osds/{0-distro/centos_8.2_container_tools_3.0', 'orch:cephadm/osds/{0-distro/centos_8.3_container_tools_3.0', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/rbd_iscsi/{0-single-container-host', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools}', 'orch:cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0', 'orch:cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0', 'orch:cephadm/thrash/{0-distro/centos_8.3_container_tools_3.0', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9', 'orch:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0', 'orch:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun', 'orchestrator_cli}', 'osd}', 'overrides/{pg-warn', 'roles', 'root}', 'start', 'start}', 'syntax', 'tasks/rados_api_tests}', 'tasks/rados_python}', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/cephadm_iscsi}']
2022-02-04T08:44:34.360 INFO:scrape:
2022-02-04T08:44:34.360 INFO:scrape:Failure: Test failure: test_cephfs_mirror (tasks.cephadm_cases.test_cli.TestCephadmCLI)
2022-02-04T08:44:34.361 INFO:scrape:1 jobs: ['6661185']
2022-02-04T08:44:34.361 INFO:scrape:suites: ['mon_election/classic', 'orch:cephadm/workunits/{agent/on', 'task/test_orch_cli}']
2022-02-04T08:44:34.361 INFO:scrape:
2022-02-04T08:44:34.362 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi085 with status 126: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1949d660777cd1fd131ec4c128a39c3342a92766 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'
2022-02-04T08:44:34.362 INFO:scrape:1 jobs: ['6661150']
2022-02-04T08:44:34.363 INFO:scrape:suites: ['mon_election/connectivity', 'orch:cephadm/workunits/{agent/off', 'task/test_cephadm}']
2022-02-04T08:44:34.363 INFO:scrape:
2022-02-04T08:44:34.366 INFO:teuthology.results:Sending results to mfritch@suse.com: Test Run: mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/
failed:  62
dead:    0
running: 0
waiting: 0
queued:  0
passed:  13


Fail
=================================================================
[6661131]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661131/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661131/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e35ec2e3413942f2becea26bfa19cfc6

    Command failed on smithi190 with status 1: 'sudo yum install -y kernel'

[6661133]  orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
-----------------------------------------------------------------
time:   00:03:32
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661133/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661133/
sentry: https://sentry.ceph.com/organizations/ceph/?query=aa221137deb54afda4378cb3d13ad8f9

    Command failed on smithi067 with status 5: 'sudo systemctl stop
    ceph-0b90140c-858f-11ec-8c35-001a4aab830c@mon.a'

[6661134]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
-----------------------------------------------------------------
time:   00:10:03
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661134/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661134/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bb6a1381de31452ea5bd19a8a225529f

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661135]  orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all}
-----------------------------------------------------------------
time:   00:10:22
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661135/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661135/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ca2a3c62ba19412ea53edb47fdb2fe56

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661136]  orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661136/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661136/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e31dd8be84314259bb894e7e29cec896

    Command failed on smithi109 with status 1: 'sudo yum install -y kernel'

[6661137]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661137/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661137/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f75ce23f983441779309f3e4045d5159

    Command failed on smithi134 with status 1: 'sudo yum install -y kernel'

[6661138]  orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/on orchestrator_cli}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661138/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661138/
sentry: https://sentry.ceph.com/organizations/ceph/?query=de71ca01258c4eb38480caad0eb4378b

    Command failed on smithi116 with status 1: 'sudo yum install -y kernel'

[6661139]  orch:cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661139/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661139/
sentry: https://sentry.ceph.com/organizations/ceph/?query=43a0041ee6fa4fdd9781f353c23b7458

    Command failed on smithi045 with status 1: 'sudo yum install -y kernel'

[6661140]  orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661140/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661140/
sentry: https://sentry.ceph.com/organizations/ceph/?query=61c5c61638864149a7c90a30bec6e714

    Command failed on smithi183 with status 1: 'sudo yum install -y kernel'

[6661142]  orch:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661142/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661142/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d1ba447f19924bb9a3fb7d7552df57cd

    Command failed on smithi148 with status 1: 'sudo yum install -y kernel'

[6661143]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final}
-----------------------------------------------------------------
time:   00:10:28
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661143/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661143/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ba7e8f0354b446479585712d01d57b5d

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661145]  orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661145/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661145/
sentry: https://sentry.ceph.com/organizations/ceph/?query=144b11e315ad49f5bf2d99a01fbf532b

    Command failed on smithi119 with status 1: 'sudo yum install -y kernel'

[6661147]  orch:cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661147/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661147/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7fd09b66af124c18b92c344ebf120d2b

    Command failed on smithi189 with status 1: 'sudo yum install -y kernel'

[6661148]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661148/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661148/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5a8a05a4e1f14e178b8c4d16e080affd

    Command failed on smithi173 with status 1: 'sudo yum install -y kernel'

[6661149]  orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661149/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661149/
sentry: https://sentry.ceph.com/organizations/ceph/?query=944c08c5cff8450c84eaabbc3f684a1e

    Command failed on smithi096 with status 1: 'sudo yum install -y kernel'

[6661150]  orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm}
-----------------------------------------------------------------
time:   00:13:43
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661150/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661150/
sentry: https://sentry.ceph.com/organizations/ceph/?query=985f33a938d3479db10f233e316825bd

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi085 with
    status 126: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=1949d660777cd1fd131ec4c128a39c3342a92766
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[6661151]  orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661151/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661151/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b02594d0d4464248aaccae35943511e6

    Command failed on smithi190 with status 1: 'sudo yum install -y kernel'

[6661152]  orch:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661152/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661152/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4accf5a94bad481b9537f455213d8fb7

    Command failed on smithi125 with status 1: 'sudo yum install -y kernel'

[6661153]  orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic}
-----------------------------------------------------------------
time:   00:03:33
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661153/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661153/
sentry: https://sentry.ceph.com/organizations/ceph/?query=730394cffc6f4caf977b3338292e194b

    Command failed on smithi045 with status 5: 'sudo systemctl stop
    ceph-350a9518-8590-11ec-8c35-001a4aab830c@mon.a'

[6661154]  orch:cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661154/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661154/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1183586cb5b04780b89ecd1abbe396ff

    Command failed on smithi181 with status 1: 'sudo yum install -y kernel'

[6661155]  orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root}
-----------------------------------------------------------------
time:   00:21:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661155/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661155/
sentry: https://sentry.ceph.com/organizations/ceph/?query=25e3ae1be30e4dddb1529ea298a00c67

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661156]  orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661156/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661156/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4a45fe028720423b9011ca9e8d28f3bc

    Command failed on smithi116 with status 1: 'sudo yum install -y kernel'

[6661157]  orch:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661157/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661157/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fcf75677c86a4ea0bd9390a5d5ff5156

    Command failed on smithi031 with status 1: 'sudo yum install -y kernel'

[6661158]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661158/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661158/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4542649659584afd94d3c835a5d49e1b

    Command failed on smithi105 with status 1: 'sudo yum install -y kernel'

[6661159]  orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661159/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661159/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cc53d69577ea4060992299cd14171d53

    Command failed on smithi142 with status 1: 'sudo yum install -y kernel'

[6661161]  orch:cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661161/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661161/
sentry: https://sentry.ceph.com/organizations/ceph/?query=953ea8b203da41d6b5b47b054b5db523

    Command failed on smithi131 with status 1: 'sudo yum install -y kernel'

[6661162]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661162/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661162/
sentry: https://sentry.ceph.com/organizations/ceph/?query=35879fe1259240f3a1d62cf73f2e0176

    Command failed on smithi121 with status 1: 'sudo yum install -y kernel'

[6661163]  orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:20:01
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661163/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661163/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8ebc6de832174346ae8d55d06166d126

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661164]  orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661164/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661164/
sentry: https://sentry.ceph.com/organizations/ceph/?query=969c6517644d41eea16a0cf2ec3a8923

    Command failed on smithi161 with status 1: 'sudo yum install -y kernel'

[6661165]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661165/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661165/
sentry: https://sentry.ceph.com/organizations/ceph/?query=633c7caca56f4438952167f260640139

    Command failed on smithi140 with status 1: 'sudo yum install -y kernel'

[6661167]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/client-keyring 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661167/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661167/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4ba99edd3b304190a3ea73acf46f4730

    Command failed on smithi196 with status 1: 'sudo yum install -y kernel'

[6661168]  orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661168/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661168/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5716ba45e2d44692a56d21f50ef2ccde

    Command failed on smithi135 with status 1: 'sudo yum install -y kernel'

[6661169]  orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661169/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661169/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dd000b1768cf4aa6b145fc5a1fdb3d36

    Command failed on smithi170 with status 1: 'sudo yum install -y kernel'

[6661170]  orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:09:58
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661170/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661170/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bec0718363634ef1a28af88cbad4ac15

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661173]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final}
-----------------------------------------------------------------
time:   00:09:58
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661173/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661173/
sentry: https://sentry.ceph.com/organizations/ceph/?query=08532e9895184672896d9c5b73490343

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661174]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661174/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661174/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2397cec0076c4d618e39d4f0978f4664

    Command failed on smithi181 with status 1: 'sudo yum install -y kernel'

[6661175]  orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661175/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661175/
sentry: https://sentry.ceph.com/organizations/ceph/?query=af535231623f4b1a9ea2ca63e860dc0e

    Command failed on smithi195 with status 1: 'sudo yum install -y kernel'

[6661176]  orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.2_container_tools_3.0} 2-node-mgr agent/off orchestrator_cli}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661176/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661176/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d82772ce043d4d048fe6854a07585c75

    Command failed on smithi082 with status 1: 'sudo yum install -y kernel'

[6661177]  orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661177/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661177/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5e8fb2e5bec0410aa0292bd1c1ecb26a

    Command failed on smithi109 with status 1: 'sudo yum install -y kernel'

[6661178]  orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661178/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661178/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cfc0ab54b8414dc8a9c72f31316ba767

    Command failed on smithi125 with status 1: 'sudo yum install -y kernel'

[6661179]  orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:20:09
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661179/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661179/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e8ea7a7f63e44ccb99c85e3693e52064

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661180]  orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final}
-----------------------------------------------------------------
time:   00:10:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661180/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661180/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e51fdd436726420bade21ec7437fbd44

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661181]  orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661181/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661181/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4fc3fec6808a4c08b5afe7ec212d3a6d

    Command failed on smithi166 with status 1: 'sudo yum install -y kernel'

[6661182]  orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:09:53
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661182/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661182/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0fd062df071448ff9b7f9c4207c89d35

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661183]  orch:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661183/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661183/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bcfbc57e1cec40ee98e12814bb241794

    Command failed on smithi067 with status 1: 'sudo yum install -y kernel'

[6661185]  orch:cephadm/workunits/{agent/on mon_election/classic task/test_orch_cli}
-----------------------------------------------------------------
time:   00:12:31
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661185/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661185/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7ea18f6e79ba46aaa33862cc9327a507

    Test failure: test_cephfs_mirror
    (tasks.cephadm_cases.test_cli.TestCephadmCLI)

[6661186]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661186/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661186/
sentry: https://sentry.ceph.com/organizations/ceph/?query=42f7f0f0bdff4e2189c80bb10a6e924e

    Command failed on smithi176 with status 1: 'sudo yum install -y kernel'

[6661189]  orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:09:57
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661189/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661189/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eecabedd091d4fda941700daf59c9b83

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661190]  orch:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661190/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661190/
sentry: https://sentry.ceph.com/organizations/ceph/?query=871efe518af347dfb4b02ba7da7258aa

    Command failed on smithi201 with status 1: 'sudo yum install -y kernel'

[6661191]  orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic}
-----------------------------------------------------------------
time:   00:03:31
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661191/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661191/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1125e0bb9eb14d0c97866360868c1a79

    Command failed on smithi026 with status 5: 'sudo systemctl stop ceph-
    ba0534ca-8591-11ec-8c35-001a4aab830c@mon.a'

[6661192]  orch:cephadm/thrash/{0-distro/centos_8.3_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661192/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661192/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c426e3e0dfcd444eb3ddcb6e3181f5f8

    Command failed on smithi121 with status 1: 'sudo yum install -y kernel'

[6661193]  orch:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661193/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661193/
sentry: https://sentry.ceph.com/organizations/ceph/?query=db29aac60cc4476caecc1777b1ba46ea

    Command failed on smithi152 with status 1: 'sudo yum install -y kernel'

[6661194]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661194/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661194/
sentry: https://sentry.ceph.com/organizations/ceph/?query=453884a88d404f329d7616b17855b3cb

    Command failed on smithi197 with status 1: 'sudo yum install -y kernel'

[6661195]  orch:cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661195/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661195/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9d9b8991acef4a7488595d788b07258b

    Command failed on smithi016 with status 1: 'sudo yum install -y kernel'

[6661197]  orch:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661197/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661197/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8695e74636de4319afbc77baecc8717d

    Command failed on smithi027 with status 1: 'sudo yum install -y kernel'

[6661198]  orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag}
-----------------------------------------------------------------
time:   00:09:59
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661198/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661198/
sentry: https://sentry.ceph.com/organizations/ceph/?query=464e7aab2a774a58aaf9e61041ffe22a

    no results found at https://shaman.ceph.com/api/search/?project=ceph&distro
    s=rhel%2Footpa%2Fx86_64&flavor=default&sha1=1949d660777cd1fd131ec4c128a39c3
    342a92766

[6661199]  orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661199/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661199/
sentry: https://sentry.ceph.com/organizations/ceph/?query=783f983858894032afd20f8771c1f1d0

    Command failed on smithi140 with status 1: 'sudo yum install -y kernel'

[6661200]  orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661200/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661200/
sentry: https://sentry.ceph.com/organizations/ceph/?query=decc2bcf7e2b41cb88fca60d410e83d0

    Command failed on smithi071 with status 1: 'sudo yum install -y kernel'

[6661201]  orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661201/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661201/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ed05c84fcaad401f958395eb8daf43ae

    Command failed on smithi027 with status 1: 'sudo yum install -y kernel'

[6661202]  orch:cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661202/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661202/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e284c84faf664ba185e0062e8d1356ce

    Command failed on smithi157 with status 1: 'sudo yum install -y kernel'

[6661204]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661204/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661204/
sentry: https://sentry.ceph.com/organizations/ceph/?query=684bc033ac2e4c32b9280aa4c394f7b5

    Command failed on smithi083 with status 1: 'sudo yum install -y kernel'

[6661205]  orch:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661205/
log:    http://qa-proxy.ceph.com/teuthology/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661205/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ba45208a191b46928e6d6e43bb4f20d7

    Command failed on smithi111 with status 1: 'sudo yum install -y kernel'



Pass
=================================================================
[6661132] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:17:30
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661132/

[6661141] orch:cephadm/workunits/{agent/on mon_election/classic task/test_adoption}
time:   00:07:01
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661141/

[6661144] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:13:47
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661144/

[6661146] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:12:49
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661146/

[6661160] orch:cephadm/workunits/{agent/on mon_election/classic task/test_cephadm_repos}
time:   00:04:13
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661160/

[6661166] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python}
time:   00:35:47
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661166/

[6661171] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root}
time:   00:40:14
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661171/

[6661172] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_nfs}
time:   00:23:28
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661172/

[6661184] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:15:18
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661184/

[6661187] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python}
time:   00:25:45
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661187/

[6661188] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:17:44
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661188/

[6661196] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_adoption}
time:   00:07:04
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661196/

[6661203] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:12:01
info:   http://pulpito.ceph.com/mgfritch-2022-02-04_07:39:29-orch:cephadm-wip-mgfritch-testing-2022-02-03-2116-distro-default-smithi/6661203/